Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.301194944s FAIL (2025-08-08 08:42:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.301228046s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 836.548051ms FAIL (2025-08-08 08:41:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 836.58561ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.034636872s FAIL (2025-08-08 08:41:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 1.034668018s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.84121032s FAIL (2025-08-08 08:41:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.84125418s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.669550388s FAIL (2025-08-08 08:40:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 1.669577043s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.436774923s FAIL (2025-08-08 08:40:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.436816858s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.400887227s FAIL (2025-08-08 08:40:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.400920876s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 850.319534ms FAIL (2025-08-08 08:39:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 850.369552ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 863.513316ms FAIL (2025-08-08 08:38:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 863.542307ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.345886146s FAIL (2025-08-08 08:38:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.34592218s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 824.321994ms FAIL (2025-08-08 08:38:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 824.342982ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.344285773s FAIL (2025-08-08 08:38:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.344318726s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 863.736214ms FAIL (2025-08-08 08:37:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 863.771174ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 833.754227ms FAIL (2025-08-08 08:37:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 833.805607ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.237430411s FAIL (2025-08-08 08:37:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.237464548s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 867.008537ms FAIL (2025-08-08 08:36:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 867.06629ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.361835732s FAIL (2025-08-08 08:36:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.361871276s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.387398372s FAIL (2025-08-08 08:36:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.387500392s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 862.72877ms FAIL (2025-08-08 08:35:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 862.755973ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.363928045s FAIL (2025-08-08 08:35:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.363960882s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.482814796s FAIL (2025-08-08 08:35:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.482849582s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 826.897969ms FAIL (2025-08-08 08:34:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 826.932577ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.348074298s FAIL (2025-08-08 08:34:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.348151898s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.006692404s FAIL (2025-08-08 08:33:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 1.006716786s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.496827721s FAIL (2025-08-08 08:33:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.496935861s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 830.541911ms FAIL (2025-08-08 08:33:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 830.594394ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 849.058954ms FAIL (2025-08-08 08:32:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 849.11291ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X......................................X....................] duration: 2.319569402s FAIL (2025-08-08 08:32:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.319601797s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X......................................X...................] duration: 2.397451329s FAIL (2025-08-08 08:31:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.397487441s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 853.2529ms FAIL (2025-08-08 08:31:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 853.283136ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X......................................X..................] duration: 2.345054928s FAIL (2025-08-08 08:31:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.345086577s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 839.828157ms FAIL (2025-08-08 08:30:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 839.864552ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X......................................X.................] duration: 2.320550032s FAIL (2025-08-08 08:30:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.320584715s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 827.898025ms FAIL (2025-08-08 08:29:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 827.930798ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X......................................X................] duration: 2.316390545s FAIL (2025-08-08 08:29:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.316432452s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.015107017s FAIL (2025-08-08 08:28:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 1.015140785s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X......................................X...............] duration: 2.329393802s FAIL (2025-08-08 08:28:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.329430983s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 845.642616ms FAIL (2025-08-08 08:28:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 845.676265ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X......................................X..............] duration: 2.357080364s FAIL (2025-08-08 08:27:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.357135942s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.168311134s FAIL (2025-08-08 08:27:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 1.168347442s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X......................................X.............] duration: 2.372048607s FAIL (2025-08-08 08:26:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.372084922s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X......................................X............] duration: 2.406155693s FAIL (2025-08-08 08:26:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.406193622s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.118258507s FAIL (2025-08-08 08:26:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 1.118287653s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 837.548996ms FAIL (2025-08-08 08:26:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 837.581632ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X......................................X...........] duration: 2.471039156s FAIL (2025-08-08 08:25:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.471079458s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 929.069647ms FAIL (2025-08-08 08:25:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 929.138796ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.51890222s FAIL (2025-08-08 08:24:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 1.518939017s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X......................................X..........] duration: 2.540351s FAIL (2025-08-08 08:24:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.540407901s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 818.717414ms FAIL (2025-08-08 08:23:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== loading segments... Aug 8 08:09:32.794169624 (22h29m54.538226473s) [upload=23276]: Upload finished. err: <nil> Aug 8 08:10:29.836939612 (22h30m51.580996462s) [upload=23299]: Starting upload Aug 8 08:10:29.837025487 (22h30m51.581082338s) [upload=23299]: Done starting upload Aug 8 08:10:29.837164063 (22h30m51.581220931s) [upload=23299]: Uploading segments... Aug 8 08:10:29.837229228 (22h30m51.58128609s) [upload=23299]: Got next segment. Inline: true Aug 8 08:10:29.837234342 (22h30m51.581291201s) [upload=23299]: Waiting for error group managing segments... Aug 8 08:10:29.837275114 (22h30m51.581331962s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:29.837279640 (22h30m51.5813365s) [upload=23299]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:29.837282167 (22h30m51.581339027s) [upload=23299]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:30.221609733 (22h30m51.965666588s) [upload=23299]: Done uploading segments... Aug 8 08:10:30.221635147 (22h30m51.965691999s) [upload=23299]: Upload finished. err: <nil> Aug 8 08:10:46.554914705 (22h31m8.298971575s) [upload=23306]: Starting upload Aug 8 08:10:46.555035271 (22h31m8.299092127s) [upload=23306]: Done starting upload Aug 8 08:10:46.555155837 (22h31m8.299212701s) [upload=23306]: Uploading segments... Aug 8 08:10:46.555285312 (22h31m8.299342188s) [upload=23306]: Got next segment. Inline: true Aug 8 08:10:46.555294037 (22h31m8.29935091s) [upload=23306]: Waiting for error group managing segments... Aug 8 08:10:46.555373576 (22h31m8.299430446s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:10:46.555380833 (22h31m8.299437703s) [upload=23306]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:10:46.555388232 (22h31m8.299445104s) [upload=23306]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:10:46.709433834 (22h31m8.453490683s) [upload=23306]: Done uploading segments... Aug 8 08:10:46.709487101 (22h31m8.453543949s) [upload=23306]: Upload finished. err: <nil> Aug 8 08:11:10.552157402 (22h31m32.296214269s) [upload=23313]: Starting upload Aug 8 08:11:10.552442618 (22h31m32.296499493s) [upload=23313]: Done starting upload Aug 8 08:11:10.552586700 (22h31m32.296643566s) [upload=23313]: Uploading segments... Aug 8 08:11:10.552692759 (22h31m32.296749636s) [upload=23313]: Got next segment. Inline: true Aug 8 08:11:10.552701484 (22h31m32.296758367s) [upload=23313]: Waiting for error group managing segments... Aug 8 08:11:10.552814800 (22h31m32.296871662s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:10.552824977 (22h31m32.296881843s) [upload=23313]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:10.552829160 (22h31m32.296886025s) [upload=23313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:10.809107506 (22h31m32.553164367s) [upload=23313]: Done uploading segments... Aug 8 08:11:10.809128806 (22h31m32.553185656s) [upload=23313]: Upload finished. err: <nil> Aug 8 08:11:51.665066000 (22h32m13.409122864s) [upload=23327]: Starting upload Aug 8 08:11:51.665134679 (22h32m13.409191538s) [upload=23327]: Done starting upload Aug 8 08:11:51.665212446 (22h32m13.409269303s) [upload=23327]: Uploading segments... Aug 8 08:11:51.665271977 (22h32m13.409328847s) [upload=23327]: Got next segment. Inline: true Aug 8 08:11:51.665280321 (22h32m13.40933719s) [upload=23327]: Waiting for error group managing segments... Aug 8 08:11:51.665336951 (22h32m13.409393801s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:11:51.665342944 (22h32m13.409399805s) [upload=23327]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:11:51.665346833 (22h32m13.409403695s) [upload=23327]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:11:51.825733106 (22h32m13.56978998s) [upload=23327]: Done uploading segments... Aug 8 08:11:51.825774704 (22h32m13.569831581s) [upload=23327]: Upload finished. err: <nil> Aug 8 08:12:36.713629283 (22h32m58.457686133s) [upload=23340]: Starting upload Aug 8 08:12:36.713677793 (22h32m58.45773466s) [upload=23340]: Done starting upload Aug 8 08:12:36.713768858 (22h32m58.457825706s) [upload=23340]: Uploading segments... Aug 8 08:12:36.713810753 (22h32m58.457867622s) [upload=23340]: Got next segment. Inline: true Aug 8 08:12:36.713816380 (22h32m58.457873243s) [upload=23340]: Waiting for error group managing segments... Aug 8 08:12:36.713871367 (22h32m58.457928216s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:12:36.713876878 (22h32m58.457933741s) [upload=23340]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:12:36.713880179 (22h32m58.45793704s) [upload=23340]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:12:36.860375743 (22h32m58.604432607s) [upload=23340]: Done uploading segments... Aug 8 08:12:36.860409095 (22h32m58.60446596s) [upload=23340]: Upload finished. err: <nil> Aug 8 08:13:34.010890464 (22h33m55.754947314s) [upload=23357]: Starting upload Aug 8 08:13:34.010944798 (22h33m55.755001648s) [upload=23357]: Done starting upload Aug 8 08:13:34.011030226 (22h33m55.755087078s) [upload=23357]: Uploading segments... Aug 8 08:13:34.011071681 (22h33m55.755128547s) [upload=23357]: Got next segment. Inline: true Aug 8 08:13:34.011076449 (22h33m55.755133302s) [upload=23357]: Waiting for error group managing segments... Aug 8 08:13:34.011120225 (22h33m55.755177076s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:13:34.011126537 (22h33m55.755183388s) [upload=23357]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:13:34.011132794 (22h33m55.755189644s) [upload=23357]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:13:34.169754204 (22h33m55.913811067s) [upload=23357]: Done uploading segments... Aug 8 08:13:34.169777081 (22h33m55.913833941s) [upload=23357]: Upload finished. err: <nil> Aug 8 08:14:05.431776093 (22h34m27.175832955s) [upload=23367]: Starting upload Aug 8 08:14:05.431855898 (22h34m27.175912767s) [upload=23367]: Done starting upload Aug 8 08:14:05.431939997 (22h34m27.175996849s) [upload=23367]: Uploading segments... Aug 8 08:14:05.431995611 (22h34m27.17605248s) [upload=23367]: Got next segment. Inline: true Aug 8 08:14:05.432001071 (22h34m27.176057931s) [upload=23367]: Waiting for error group managing segments... Aug 8 08:14:05.432063159 (22h34m27.17612001s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:14:05.432070467 (22h34m27.176127317s) [upload=23367]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:14:05.432074258 (22h34m27.17613111s) [upload=23367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:14:05.625809853 (22h34m27.369866707s) [upload=23367]: Done uploading segments... Aug 8 08:14:05.625830031 (22h34m27.369886882s) [upload=23367]: Upload finished. err: <nil> Aug 8 08:15:07.534950807 (22h35m29.279007669s) [upload=23383]: Starting upload Aug 8 08:15:07.535020031 (22h35m29.279076889s) [upload=23383]: Done starting upload Aug 8 08:15:07.535117916 (22h35m29.279174769s) [upload=23383]: Uploading segments... Aug 8 08:15:07.535195212 (22h35m29.279252078s) [upload=23383]: Got next segment. Inline: true Aug 8 08:15:07.535200934 (22h35m29.279257806s) [upload=23383]: Waiting for error group managing segments... Aug 8 08:15:07.535540314 (22h35m29.279597177s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:07.535549666 (22h35m29.27960653s) [upload=23383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:07.535555009 (22h35m29.279611878s) [upload=23383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:07.697698439 (22h35m29.441755304s) [upload=23383]: Done uploading segments... Aug 8 08:15:07.697725357 (22h35m29.441782221s) [upload=23383]: Upload finished. err: <nil> Aug 8 08:15:42.083496980 (22h36m3.827553843s) [upload=23393]: Starting upload Aug 8 08:15:42.083555873 (22h36m3.827612724s) [upload=23393]: Done starting upload Aug 8 08:15:42.083641487 (22h36m3.827698346s) [upload=23393]: Uploading segments... Aug 8 08:15:42.083710642 (22h36m3.827767511s) [upload=23393]: Got next segment. Inline: true Aug 8 08:15:42.083717471 (22h36m3.827774331s) [upload=23393]: Waiting for error group managing segments... Aug 8 08:15:42.083765650 (22h36m3.8278225s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:42.083773936 (22h36m3.827830796s) [upload=23393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:42.083777986 (22h36m3.827834845s) [upload=23393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:42.228592880 (22h36m3.972649744s) [upload=23393]: Done uploading segments... Aug 8 08:15:42.228668918 (22h36m3.972725769s) [upload=23393]: Upload finished. err: <nil> Aug 8 08:15:58.128103092 (22h36m19.872159942s) [upload=23398]: Starting upload Aug 8 08:15:58.128175442 (22h36m19.872232303s) [upload=23398]: Done starting upload Aug 8 08:15:58.128434515 (22h36m19.872491371s) [upload=23398]: Uploading segments... Aug 8 08:15:58.128544452 (22h36m19.872601306s) [upload=23398]: Got next segment. Inline: true Aug 8 08:15:58.128552404 (22h36m19.872609253s) [upload=23398]: Waiting for error group managing segments... Aug 8 08:15:58.128628635 (22h36m19.872685501s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:58.128635719 (22h36m19.872692583s) [upload=23398]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 08:15:58.128640733 (22h36m19.872697601s) [upload=23398]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:58.217762569 (22h36m19.96181943s) [upload=23398]: Done uploading segments... Aug 8 08:15:58.217781822 (22h36m19.961838682s) [upload=23398]: Upload finished. err: metaclient: Too Many Requests Duration: 818.830764ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X......................................X.........] duration: 2.314009203s FAIL (2025-08-08 08:23:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 63c5ed61906e11936b68f2ef52fda9f63fb635ba87d64be4f33bf3a1c4bc8e75 != before 1622e88812ebafab9753f9096dd3c401166e31560ad135f8f9a49413a63f7e27 ======LOGS====== 8s) [upload=23386]: Uploading segments... Aug 8 08:15:27.387895706 (22h35m49.131952567s) [upload=23386]: Got next segment. Inline: false Aug 8 08:15:27.387970058 (22h35m49.132026908s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 08:15:27.387976701 (22h35m49.132033565s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 08:15:27.586328226 (22h35m49.330385077s) [seg_pos={0 0}, upload=23386]: Begin upload segment... Aug 8 08:15:27.586390235 (22h35m49.330447099s) [seg_pos={0 0}, upload=23386]: Done begin upload segment. Aug 8 08:15:27.586434842 (22h35m49.330491713s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586451078 (22h35m49.330507943s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.586872054 (22h35m49.330928917s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587210058 (22h35m49.331266918s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587532196 (22h35m49.331589056s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.587844962 (22h35m49.331901822s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588189974 (22h35m49.332246841s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.588643723 (22h35m49.33270059s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589042544 (22h35m49.333099405s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589382110 (22h35m49.333438973s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589694361 (22h35m49.33375123s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.589787307 (22h35m49.33384417s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590277824 (22h35m49.334334687s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590630718 (22h35m49.334687579s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.590943350 (22h35m49.335000216s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591087045 (22h35m49.335143916s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591500868 (22h35m49.335557727s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:27.591842841 (22h35m49.335899701s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Uploading piece... Aug 8 08:15:28.158331873 (22h35m49.902388733s) [upload=23386]: Next returned nil segment Aug 8 08:15:28.158353292 (22h35m49.902410143s) [upload=23386]: Waiting for error group managing segments... Aug 8 08:15:28.322414292 (22h35m50.066471145s) [noise=true , address=45.91.116.8:4059 , piece_id=NHQAF7GJ4NRQUCS53RZN5DCV366HJSFCOMCGVU7ME5ICRR64O7WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.323117212 (22h35m50.067174063s) [noise=true , address=45.91.116.14:4204 , piece_id=DALND2AOMMJMY5GDDRNLTXK3PX2JLFUMXZ5Y4HUJGQIZ5SIMXOFA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328133673 (22h35m50.072190523s) [noise=true , address=104.36.82.62:4416 , piece_id=ZKCNR4NAZGHJ52SYI6Q6ZPW5QNCD4GNY25QKUQXTYMMI2CCV6HLA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.328661020 (22h35m50.07271788s) [noise=true , address=45.91.116.13:4168 , piece_id=UACG2O5A4OU6I2ERIOPNDWCSVKTPRVXZMGMJHFBGQTDMIJAHLBGQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340035703 (22h35m50.084092555s) [noise=true , address=104.36.82.45:4926 , piece_id=Y2XCSLAA4TCZCE2YJM5NT5ZY4DODJ6MQQAGPFFGK7I323WH66QVA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.340562308 (22h35m50.084619159s) [noise=true , address=45.91.116.7:4034 , piece_id=5ABEJBJTIWU77AUFGRVUPN5FIHE7LAU4P6PRTR7EEUTW7JXSN6TA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.341044839 (22h35m50.085101689s) [noise=true , address=104.36.82.61:4408 , piece_id=XGZ233VZ4ZPWDQKIOEXSXH3WO53WQD6KUPOSI7CBWJABEBJPG4AA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.342424182 (22h35m50.086481029s) [noise=true , address=104.36.82.14:4253 , piece_id=M32GZERPRPZIEKQGMUOA2FQSAMIZBYLXULL3KNBN7XNFR2KGG6GQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.353086606 (22h35m50.09714346s) [noise=true , address=104.36.82.46:4486 , piece_id=P6XNH6SPESJH5XJCEYQCACEXU5QRFMAUTGPVWO4RBSUOWYX32J6Q, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354242357 (22h35m50.098299209s) [noise=true , address=104.36.82.23:4469 , piece_id=OA7653VXCPN3KBZWU644UJPISLYZ6OS764PDWPLMVBLVWNAYUVVQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.354709810 (22h35m50.098766673s) [noise=true , address=104.36.82.60:4371 , piece_id=UGOCJU4NOWGW73AHXPBXHYFJLWGEVCT7MHCBHYIRITJQDPKHMBOQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.359072209 (22h35m50.103129061s) [noise=true , address=104.36.82.37:4661 , piece_id=JPSASPR3BIZ7QQW7KHBBTLH3SLIYM2ARLADOYTT2DI637ILDU5WA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.368405498 (22h35m50.112462344s) [noise=true , address=104.36.82.59:4303 , piece_id=ACINHYN7D6XRV5NTBN6CE4536ZUPSMWGZGZZJVUJBCIHDPQ7KNYA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369039702 (22h35m50.113096548s) [noise=true , address=104.36.82.16:4315 , piece_id=GYYIGVQNWW2OWKVXFIYWQI4ZWEKAU7TNCGRA55FGPKMJCKCVZBGA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: <nil> Aug 8 08:15:28.369050850 (22h35m50.113107696s) [seg_pos={0 0}, upload=23386]: Segment reached optimal threshold of 14 pieces. Aug 8 08:15:28.369182993 (22h35m50.113239839s) [noise=true , address=104.36.82.35:4599 , piece_id=PC66PXZI4Q7UH2QZKC5BXR5GU5CK77JN5IFHUUZUKKLRURQB6ACA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxKQjpzyhZsRnkffX1fwCXcTYiwHPwuNg4HWtpdzRnKQCtp6s): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369466440 (22h35m50.113523287s) [noise=true , address=104.36.82.4:4836 , piece_id=VO5TFY45CY2SXRLKETIRDWQ5P4GNNXEJRYESVABHJ2AVHUOWOMDQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369530522 (22h35m50.113587368s) [noise=true , address=104.36.82.44:4876 , piece_id=ALYV7HCNJG4TFI5D4VOX3OZTNXSWBQIRMVU4EVKVIDGMMQLWOIUQ, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RZEXg8h3qbrw3mJadmpudtBsVadHMb4knwtueL7GhdHLkESYC): protocol: expected piece hash; context canceled; EOF Aug 8 08:15:28.369638823 (22h35m50.11369567s) [noise=true , address=64.49.13.169:4710 , piece_id=PPWQZEVYYT3N6CUYZ4CA3JK4VLHANSEHOBVMD2C7YZWUK2F54LRA, seg_pos={0 0}, upload=23386]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36670->64.49.13.169:4710: use of closed network connection Aug 8 08:15:28.369651872 (22h35m50.113708719s) [seg_pos={0 0}, upload=23386]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 08:15:28.369716021 (22h35m50.113772882s) [upload=23386]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 08:15:28.369722390 (22h35m50.11377925s) [upload=23386]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 08:15:28.517255405 (22h35m50.261312275s) [upload=23386]: Done uploading segments... Aug 8 08:15:28.517280948 (22h35m50.261337812s) [upload=23386]: Upload finished. err: <nil> Duration: 2.31404271s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)