Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.975628368s FAIL (2025-08-19 07:00:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.975671424s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 790.905627ms FAIL (2025-08-19 06:59:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 790.942613ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 870.316072ms FAIL (2025-08-19 06:59:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 870.416397ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.323655004s FAIL (2025-08-19 06:59:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.323861006s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 809.008762ms FAIL (2025-08-19 06:58:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 809.042403ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.789763659s FAIL (2025-08-19 06:58:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.789853978s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 853.259912ms FAIL (2025-08-19 06:58:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 853.316879ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X..........................X................................] duration: 833.037434ms FAIL (2025-08-19 06:58:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 833.087724ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 832.105365ms FAIL (2025-08-19 06:57:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 832.138374ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X..........................X...............................] duration: 830.659229ms FAIL (2025-08-19 06:57:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 830.776792ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.361167179s FAIL (2025-08-19 06:57:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.361247378s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 848.238912ms FAIL (2025-08-19 06:57:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 848.273484ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X..........................X..............................] duration: 993.224067ms FAIL (2025-08-19 06:56:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 993.346731ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 855.998298ms FAIL (2025-08-19 06:56:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 856.030872ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.465357269s FAIL (2025-08-19 06:56:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.465392301s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.398880607s FAIL (2025-08-19 06:55:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.398913528s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X..........................X.............................] duration: 822.255754ms FAIL (2025-08-19 06:55:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 822.2889ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.052563582s FAIL (2025-08-19 06:55:09 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.052588546s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.684598108s FAIL (2025-08-19 06:55:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.684637711s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 839.003739ms FAIL (2025-08-19 06:54:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 839.031148ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X..........................X............................] duration: 863.826243ms FAIL (2025-08-19 06:54:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 863.858785ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.51879568s FAIL (2025-08-19 06:54:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.518905766s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.018545547s FAIL (2025-08-19 06:54:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.018588709s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X..........................X...........................] duration: 815.049608ms FAIL (2025-08-19 06:53:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 815.089779ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.473648808s FAIL (2025-08-19 06:53:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.473689213s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 855.220658ms FAIL (2025-08-19 06:53:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 855.253439ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X..........................X..........................] duration: 828.207834ms FAIL (2025-08-19 06:52:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 828.244926ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.527138458s FAIL (2025-08-19 06:52:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.527175648s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 849.002123ms FAIL (2025-08-19 06:52:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 849.053409ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.325169827s FAIL (2025-08-19 06:51:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.325204862s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X..........................X.........................] duration: 811.749387ms FAIL (2025-08-19 06:51:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 811.787948ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1.870927187s FAIL (2025-08-19 06:51:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 1.870974862s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 833.147692ms FAIL (2025-08-19 06:50:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 833.205094ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X..........................X........................] duration: 800.548973ms FAIL (2025-08-19 06:50:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 800.653792ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 835.376696ms FAIL (2025-08-19 06:49:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 835.409159ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X..........................X.......................] duration: 947.21112ms FAIL (2025-08-19 06:49:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 947.289038ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.001179031s FAIL (2025-08-19 06:49:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.001209388s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 957.920349ms FAIL (2025-08-19 06:48:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 957.963856ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X..........................X......................] duration: 1.07892278s FAIL (2025-08-19 06:48:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.078957429s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.11089097s FAIL (2025-08-19 06:48:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.110932263s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X..........................X.....................] duration: 1.115352922s FAIL (2025-08-19 06:47:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.115384352s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 894.68159ms FAIL (2025-08-19 06:47:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 894.717667ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X..........................X....................] duration: 811.409582ms FAIL (2025-08-19 06:46:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 811.446292ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 850.704839ms FAIL (2025-08-19 06:46:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 850.759522ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X..........................X...................] duration: 896.148966ms FAIL (2025-08-19 06:45:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 896.184469ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 866.86511ms FAIL (2025-08-19 06:45:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 866.922276ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 830.951864ms FAIL (2025-08-19 06:44:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 830.983309ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X..........................X..................] duration: 1.099857354s FAIL (2025-08-19 06:44:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.099896263s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X..........................X.................] duration: 1.013825508s FAIL (2025-08-19 06:43:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.01385693s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X..........................X................] duration: 818.775223ms FAIL (2025-08-19 06:43:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 06:31:13.926153326 (253h5m35.885250565s) [upload=263460]: Starting upload Aug 19 06:31:13.926270390 (253h5m35.885367636s) [upload=263460]: Done starting upload Aug 19 06:31:13.926369925 (253h5m35.885467166s) [upload=263460]: Uploading segments... Aug 19 06:31:13.926437753 (253h5m35.885535003s) [upload=263460]: Got next segment. Inline: true Aug 19 06:31:13.926444406 (253h5m35.885541653s) [upload=263460]: Waiting for error group managing segments... Aug 19 06:31:13.926512263 (253h5m35.88560951s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:31:13.926518167 (253h5m35.885615414s) [upload=263460]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:31:13.926522412 (253h5m35.885619659s) [upload=263460]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:31:14.017173910 (253h5m35.976271184s) [upload=263460]: Done uploading segments... Aug 19 06:31:14.017197149 (253h5m35.976294413s) [upload=263460]: Upload finished. err: metaclient: Too Many Requests Duration:* 818.806688ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)