Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.340216548s FAIL (2025-08-19 07:12:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.340250088s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X............................................X.......] duration: 801.102098ms FAIL (2025-08-19 07:12:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 801.149901ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.02475684s FAIL (2025-08-19 07:12:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.024788916s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.529453448s FAIL (2025-08-19 07:12:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.529489268s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 840.254591ms FAIL (2025-08-19 07:11:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 840.288701ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.378250636s FAIL (2025-08-19 07:11:38 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.378296519s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X............................................X......] duration: 821.855178ms FAIL (2025-08-19 07:11:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 821.890946ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 805.376822ms FAIL (2025-08-19 07:10:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 805.418438ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.398096524s FAIL (2025-08-19 07:10:56 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.398193333s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 902.894854ms FAIL (2025-08-19 07:10:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 902.941871ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X............................................X.....] duration: 814.762609ms FAIL (2025-08-19 07:10:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 814.865883ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.389581208s FAIL (2025-08-19 07:09:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.389626361s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 902.220723ms FAIL (2025-08-19 07:09:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 902.259009ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.360991445s FAIL (2025-08-19 07:09:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.361036053s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X............................................X....] duration: 820.996274ms FAIL (2025-08-19 07:09:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 821.027824ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 824.390224ms FAIL (2025-08-19 07:08:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 824.423075ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.520047734s FAIL (2025-08-19 07:08:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.520083572s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X............................................X...] duration: 851.890853ms FAIL (2025-08-19 07:08:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 851.932408ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X............................................X..] duration: 818.782238ms FAIL (2025-08-19 07:08:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 818.820355ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.008342378s FAIL (2025-08-19 07:07:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.00836991s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X............................................X.] duration: 1.020517851s FAIL (2025-08-19 07:07:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.020544102s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 831.883377ms FAIL (2025-08-19 07:07:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 831.980886ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.575082594s FAIL (2025-08-19 07:07:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.575125325s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X............................................X] duration: 359.250664ms FAIL (2025-08-19 07:07:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 359.292217ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 838.762045ms FAIL (2025-08-19 07:07:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 838.878404ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.112467771s FAIL (2025-08-19 07:06:51 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.112500006s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.356914104s FAIL (2025-08-19 07:06:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.356948042s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 866.59204ms FAIL (2025-08-19 07:06:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 866.623878ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 825.779114ms FAIL (2025-08-19 07:05:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 825.832419ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.708833911s FAIL (2025-08-19 07:05:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.708866354s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.463588356s FAIL (2025-08-19 07:05:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.463618719s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.346170045s FAIL (2025-08-19 07:05:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.346232155s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 813.873291ms FAIL (2025-08-19 07:04:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 813.978916ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 923.16335ms FAIL (2025-08-19 07:04:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 923.207951ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 832.965809ms FAIL (2025-08-19 07:04:13 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 833.024405ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.525839794s FAIL (2025-08-19 07:03:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.525870722s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 985.901739ms FAIL (2025-08-19 07:03:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 985.940563ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 973.200909ms FAIL (2025-08-19 07:03:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 973.254014ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 905.517183ms FAIL (2025-08-19 07:03:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 905.551938ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.802443521s FAIL (2025-08-19 07:03:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.802498848s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 805.483551ms FAIL (2025-08-19 07:03:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 805.507871ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 3.67096662s FAIL (2025-08-19 07:02:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 3.671024807s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.161759731s FAIL (2025-08-19 07:02:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.161796503s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.462190354s FAIL (2025-08-19 07:02:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.462249256s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 914.826333ms FAIL (2025-08-19 07:01:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 914.862142ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 3.196353188s FAIL (2025-08-19 07:01:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 3.19638884s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 903.13036ms FAIL (2025-08-19 07:01:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 903.201555ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.464634325s FAIL (2025-08-19 07:00:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.464747165s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.393115254s FAIL (2025-08-19 07:00:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.393173111s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.168038745s FAIL (2025-08-19 07:00:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.168076933s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)