Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................X.....X...X..............X...........] duration: 2.903454129s FAIL (2025-08-07 16:32:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.903485857s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 929.475509ms FAIL (2025-08-07 16:32:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 929.538283ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................X.....X...X..............X..........] duration: 2.439161991s FAIL (2025-08-07 16:31:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.439254889s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.16149297s FAIL (2025-08-07 16:31:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.161586074s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................X.....X...X..............X.........] duration: 2.525788479s FAIL (2025-08-07 16:30:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.52582279s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 959.614379ms FAIL (2025-08-07 16:30:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 959.655712ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................X.....X...X..............X........] duration: 2.423625091s FAIL (2025-08-07 16:30:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.423747386s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................X.....X...X..............X.......] duration: 2.572069229s FAIL (2025-08-07 16:29:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.572114969s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 836.582079ms FAIL (2025-08-07 16:29:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 836.60812ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 976.695927ms FAIL (2025-08-07 16:28:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 976.728571ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X...........................X.....X...X..............X......] duration: 2.398794713s FAIL (2025-08-07 16:28:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.39882925s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 832.367113ms FAIL (2025-08-07 16:28:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 832.400036ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X...........................X.....X...X..............X.....] duration: 2.473615107s FAIL (2025-08-07 16:28:09 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.473648446s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X...........................X.....X...X..............X....] duration: 2.59503009s FAIL (2025-08-07 16:27:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.595059165s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.000587302s FAIL (2025-08-07 16:27:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.000612518s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X...........................X.....X...X..............X...] duration: 2.532396051s FAIL (2025-08-07 16:26:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.53242927s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 855.968563ms FAIL (2025-08-07 16:26:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 856.001867ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 827.337208ms FAIL (2025-08-07 16:25:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 827.363639ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X...........................X.....X...X..............X..] duration: 2.355762914s FAIL (2025-08-07 16:25:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.355797037s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.13229721s FAIL (2025-08-07 16:25:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.132328539s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 995.61465ms FAIL (2025-08-07 16:25:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 995.652902ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X...........................X.....X...X..............X.] duration: 2.396892482s FAIL (2025-08-07 16:24:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 2.396935225s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 896.553343ms FAIL (2025-08-07 16:24:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 896.659111ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X...........................X.....X...X..............X] duration: 1.76526877s FAIL (2025-08-07 16:23:35 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== :34.424409141 (6h43m56.168465991s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:23:34.424414139 (6h43m56.168470999s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:23:34.663914468 (6h43m56.407971338s) [seg_pos={0 0}, upload=6997]: Begin upload segment... Aug 7 16:23:34.664000233 (6h43m56.408057097s) [seg_pos={0 0}, upload=6997]: Done begin upload segment. Aug 7 16:23:34.664074012 (6h43m56.408130882s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664702875 (6h43m56.408759746s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.664851365 (6h43m56.408908225s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665403070 (6h43m56.409459931s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.665739502 (6h43m56.40979637s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666107381 (6h43m56.410164251s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666148951 (6h43m56.410205815s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666496462 (6h43m56.410553329s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666505054 (6h43m56.410561917s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666867615 (6h43m56.410924483s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.666965087 (6h43m56.41102195s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667168287 (6h43m56.411225151s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667301739 (6h43m56.411358599s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667452478 (6h43m56.411509341s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667825786 (6h43m56.411882657s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667835904 (6h43m56.411892768s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.667874187 (6h43m56.41193105s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:34.668129298 (6h43m56.412186158s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Uploading piece... Aug 7 16:23:35.261816779 (6h43m57.005873629s) [upload=6997]: Next returned nil segment Aug 7 16:23:35.261851250 (6h43m57.005908114s) [upload=6997]: Waiting for error group managing segments... Aug 7 16:23:35.440284117 (6h43m57.18434097s) [noise=true , address=104.36.82.44:4901 , piece_id=DP2WQAAPGQ6DXBRHEWV7GNHL4AFF46QIIIC4ZJV3OJWDXPYPUYLQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.442478641 (6h43m57.18653549s) [noise=true , address=104.36.82.58:4275 , piece_id=QJCHVWB27HPUONZUAH7JZRZTDS6DC65FE33KUOFNC6KABMIHC2YA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.444618356 (6h43m57.188675202s) [noise=true , address=104.36.82.39:4739 , piece_id=4YV5BTUFXM4YC5ME3LPOHG5JY4QFGG5WNL2S3W747L57SMTUFKHQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.446841616 (6h43m57.190898469s) [noise=true , address=104.36.82.42:4824 , piece_id=G7TIPMPA3MMRJQZHAFXQE74VYTWW7SONUYTMAC2GFDIL5GI3VIDA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.448418796 (6h43m57.192475649s) [noise=true , address=104.36.82.46:4508 , piece_id=M3B2WUBZAYR3EDOGR7QWYK6I4J3G5ZYGXA2YEXGZ26IXW4W2K66A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452239080 (6h43m57.196295942s) [noise=true , address=104.36.82.23:4470 , piece_id=IGRYRBD7DC3HK3NB33PIQM6GV2IEIX4LLWOFI74GI6B5GAVVJM3A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.452763137 (6h43m57.196819987s) [noise=true , address=77.90.31.10:4523 , piece_id=UETFZ56AYNFTUKE37CBOSUY6RFIZU4IB5QQ7SBAF6XYBMBRIXQFQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.454145958 (6h43m57.198202808s) [noise=true , address=104.36.82.54:4174 , piece_id=6GKEUPU3T3MLE3HXUW7WPHVGAQBHVHK3GVGRTKG6HLDIZN57CXDQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.457927751 (6h43m57.201984612s) [noise=true , address=104.36.82.9:4039 , piece_id=DACQVXHKKLZDR2CCQED46XPRQDYMVAWIUJ3KORROZX6EOJD4AXMA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.467116844 (6h43m57.211173698s) [noise=true , address=104.36.82.8:4990 , piece_id=BYVOY3MZPZ5KNP2LQ47USO6MFD2VIATGFDHIYNGSDGBKFKICPZ5A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.470612428 (6h43m57.214669279s) [noise=true , address=104.36.82.3:4798 , piece_id=6WBUI6MWBOKZCFR66F53GC5DYCHIWODGEIGCHZUHFEUZLTWYLZZQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.493372697 (6h43m57.237429563s) [noise=true , address=104.36.82.46:4500 , piece_id=R25LZ362SAXEXYFJ6WKM5W7TY3AUAY34Y7VRMEMKKXUZE6JZPS7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.538190784 (6h43m57.282247645s) [noise=true , address=104.36.82.60:4363 , piece_id=KM7U2VHHF3VSPG5OT4BO6KEDYXUBXPZYQH3AABKYCI3E3UOV5QXQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550602510 (6h43m57.29465936s) [noise=true , address=104.36.82.55:4209 , piece_id=UVIVJOSF44ENSJBTBTQSCG5KMPLN6ER2G22VQJUOUA2IIYILK7AA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: <nil> Aug 7 16:23:35.550642846 (6h43m57.294699707s) [seg_pos={0 0}, upload=6997]: Segment reached optimal threshold of 14 pieces. Aug 7 16:23:35.550812985 (6h43m57.294869844s) [noise=true , address=104.36.82.12:4174 , piece_id=2TJPQGQVLWCP2HVQREUERHENBKRME7EJJ6GJFE7RYYNHZKV3RDKQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZqPqvh4SVpDmkpVKQXZUW48CtzV9VMeMGJoDZwA7KpNfmjfow): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.550868623 (6h43m57.294925488s) [noise=true , address=104.36.82.60:4356 , piece_id=Z3JL3I5L4LYV4ZDXV22JURA336VBUIHBBU4V6LV53WGZKIHBJZTA, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ErG54kzXSK5hXDTYZpL6X3dg26sqW2kdM6CQMLp3qMMHteir1): protocol: expected piece hash; context canceled; EOF Aug 7 16:23:35.551048361 (6h43m57.295105221s) [noise=true , address=64.49.13.137:4498 , piece_id=7RUCLVYYDYNFEBT4XQZ3T3Z6FXSM5TGG54X72EUG6XNMZ3UTUGBQ, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nh2t6FxdqbwMky99pfMDNcGXozoMYn4euWvNtThnoTvPMbpS5): protocol: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36226->64.49.13.137:4498: use of closed network connection Aug 7 16:23:35.551099022 (6h43m57.295155871s) [noise=true , address=79.127.205.233:4633 , piece_id=V3LKKOF6FGVZ6OLSW3MFUGQ64I7AOYQBB2PUH6YSEUD6JOKC2Z7A, seg_pos={0 0}, upload=6997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33566->79.127.205.233:4633: use of closed network connection Aug 7 16:23:35.551133161 (6h43m57.295190028s) [seg_pos={0 0}, upload=6997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:23:35.551234135 (6h43m57.295291005s) [upload=6997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:23:35.551240893 (6h43m57.295297768s) [upload=6997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:23:35.711384872 (6h43m57.455441721s) [upload=6997]: Done uploading segments... Aug 7 16:23:35.711403321 (6h43m57.455460171s) [upload=6997]: Upload finished. err: <nil> Duration: 1.765295343s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 845.299217ms FAIL (2025-08-07 16:23:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 845.326763ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 967.951959ms FAIL (2025-08-07 16:22:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 967.999329ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X...........................X.....X...X..............] duration: 2.463884281s FAIL (2025-08-07 16:22:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.463972667s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X...........................X.....X...X.............] duration: 2.380881168s FAIL (2025-08-07 16:21:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.380912004s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.009141241s FAIL (2025-08-07 16:21:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.009183531s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X...........................X.....X...X............] duration: 2.410321919s FAIL (2025-08-07 16:21:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.410352567s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 850.527621ms FAIL (2025-08-07 16:21:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 850.557673ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X...........................X.....X...X...........] duration: 2.767450039s FAIL (2025-08-07 16:21:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.767481895s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.046574522s FAIL (2025-08-07 16:20:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.046600576s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X...........................X.....X...X..........] duration: 2.724211237s FAIL (2025-08-07 16:19:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.724254628s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 865.087378ms FAIL (2025-08-07 16:19:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 865.117768ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.003621785s FAIL (2025-08-07 16:19:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.003714751s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X...........................X.....X...X.........] duration: 2.776779107s FAIL (2025-08-07 16:19:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.776823443s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 856.319216ms FAIL (2025-08-07 16:18:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 856.346175ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X...........................X.....X...X........] duration: 2.845416277s FAIL (2025-08-07 16:18:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.84553159s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 364.856135ms FAIL (2025-08-07 16:17:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== stItem_SegmentMakeInline Aug 7 16:09:40.616753955 (6h30m2.360810806s) [upload=6752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:09:40.775802990 (6h30m2.519859842s) [upload=6752]: Done uploading segments... Aug 7 16:09:40.775838389 (6h30m2.519895239s) [upload=6752]: Upload finished. err: <nil> Aug 7 16:10:42.231759607 (6h31m3.975816478s) [upload=6768]: Starting upload Aug 7 16:10:42.231884006 (6h31m3.975940878s) [upload=6768]: Done starting upload Aug 7 16:10:42.231979135 (6h31m3.976035981s) [upload=6768]: Uploading segments... Aug 7 16:10:42.232044220 (6h31m3.976101083s) [upload=6768]: Got next segment. Inline: true Aug 7 16:10:42.232049723 (6h31m3.976106569s) [upload=6768]: Waiting for error group managing segments... Aug 7 16:10:42.232090947 (6h31m3.976147794s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:10:42.232096269 (6h31m3.97615313s) [upload=6768]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:10:42.232108835 (6h31m3.976165696s) [upload=6768]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:10:42.442278505 (6h31m4.186335359s) [upload=6768]: Done uploading segments... Aug 7 16:10:42.442306284 (6h31m4.186363135s) [upload=6768]: Upload finished. err: <nil> Aug 7 16:11:31.667749137 (6h31m53.41180599s) [upload=6781]: Starting upload Aug 7 16:11:31.667832000 (6h31m53.411888869s) [upload=6781]: Done starting upload Aug 7 16:11:31.667942448 (6h31m53.4119993s) [upload=6781]: Uploading segments... Aug 7 16:11:31.668009192 (6h31m53.412066062s) [upload=6781]: Got next segment. Inline: true Aug 7 16:11:31.668015293 (6h31m53.412072154s) [upload=6781]: Waiting for error group managing segments... Aug 7 16:11:31.668064508 (6h31m53.412121367s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:11:31.668070003 (6h31m53.412126863s) [upload=6781]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:11:31.668073848 (6h31m53.412130708s) [upload=6781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:31.844732843 (6h31m53.588789704s) [upload=6781]: Done uploading segments... Aug 7 16:11:31.844789290 (6h31m53.58884614s) [upload=6781]: Upload finished. err: <nil> Aug 7 16:12:12.650293330 (6h32m34.394350192s) [upload=6793]: Starting upload Aug 7 16:12:12.650355805 (6h32m34.394412674s) [upload=6793]: Done starting upload Aug 7 16:12:12.650500813 (6h32m34.39455766s) [upload=6793]: Uploading segments... Aug 7 16:12:12.650630867 (6h32m34.394687737s) [upload=6793]: Got next segment. Inline: true Aug 7 16:12:12.650640003 (6h32m34.39469687s) [upload=6793]: Waiting for error group managing segments... Aug 7 16:12:12.650713892 (6h32m34.394770756s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:12.650719677 (6h32m34.394776537s) [upload=6793]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:12.650725124 (6h32m34.394781984s) [upload=6793]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:12.807983137 (6h32m34.552040015s) [upload=6793]: Done uploading segments... Aug 7 16:12:12.808026051 (6h32m34.552082919s) [upload=6793]: Upload finished. err: <nil> Aug 7 16:12:30.834349250 (6h32m52.578406112s) [upload=6799]: Starting upload Aug 7 16:12:30.834410554 (6h32m52.578467416s) [upload=6799]: Done starting upload Aug 7 16:12:30.834487363 (6h32m52.578544226s) [upload=6799]: Uploading segments... Aug 7 16:12:30.834541191 (6h32m52.578598062s) [upload=6799]: Got next segment. Inline: true Aug 7 16:12:30.834547253 (6h32m52.578604115s) [upload=6799]: Waiting for error group managing segments... Aug 7 16:12:30.834597304 (6h32m52.578654164s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:30.834603379 (6h32m52.578660242s) [upload=6799]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:12:30.834607593 (6h32m52.578664455s) [upload=6799]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:12:30.992269717 (6h32m52.736326587s) [upload=6799]: Done uploading segments... Aug 7 16:12:30.992291433 (6h32m52.736348298s) [upload=6799]: Upload finished. err: <nil> Aug 7 16:13:44.023251504 (6h34m5.767308377s) [upload=6821]: Starting upload Aug 7 16:13:44.023366744 (6h34m5.767423591s) [upload=6821]: Done starting upload Aug 7 16:13:44.023608906 (6h34m5.767665763s) [upload=6821]: Uploading segments... Aug 7 16:13:44.023709410 (6h34m5.767766283s) [upload=6821]: Got next segment. Inline: true Aug 7 16:13:44.023719636 (6h34m5.767776501s) [upload=6821]: Waiting for error group managing segments... Aug 7 16:13:44.023773027 (6h34m5.767829887s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:13:44.023779863 (6h34m5.767836723s) [upload=6821]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:13:44.023783737 (6h34m5.767840586s) [upload=6821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:13:44.190324158 (6h34m5.934381018s) [upload=6821]: Done uploading segments... Aug 7 16:13:44.190372081 (6h34m5.934428942s) [upload=6821]: Upload finished. err: <nil> Aug 7 16:14:53.942530987 (6h35m15.68658786s) [upload=6844]: Starting upload Aug 7 16:14:53.942673924 (6h35m15.686730785s) [upload=6844]: Done starting upload Aug 7 16:14:53.942756501 (6h35m15.686813366s) [upload=6844]: Uploading segments... Aug 7 16:14:53.942859287 (6h35m15.686916158s) [upload=6844]: Got next segment. Inline: true Aug 7 16:14:53.942866822 (6h35m15.686923693s) [upload=6844]: Waiting for error group managing segments... Aug 7 16:14:53.942937447 (6h35m15.686994317s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:14:53.942944846 (6h35m15.687001717s) [upload=6844]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:14:53.942947964 (6h35m15.687004834s) [upload=6844]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:14:54.089614227 (6h35m15.833671089s) [upload=6844]: Done uploading segments... Aug 7 16:14:54.089638187 (6h35m15.833695047s) [upload=6844]: Upload finished. err: <nil> Aug 7 16:15:47.816160603 (6h36m9.560217463s) [upload=6863]: Starting upload Aug 7 16:15:47.816219200 (6h36m9.560276065s) [upload=6863]: Done starting upload Aug 7 16:15:47.816297832 (6h36m9.560354678s) [upload=6863]: Uploading segments... Aug 7 16:15:47.816351442 (6h36m9.560408304s) [upload=6863]: Got next segment. Inline: true Aug 7 16:15:47.816357229 (6h36m9.560414088s) [upload=6863]: Waiting for error group managing segments... Aug 7 16:15:47.816395509 (6h36m9.560452355s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:15:47.816402183 (6h36m9.560459042s) [upload=6863]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:15:47.816404894 (6h36m9.560461753s) [upload=6863]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:15:47.966035904 (6h36m9.710092775s) [upload=6863]: Done uploading segments... Aug 7 16:15:47.966061058 (6h36m9.710117917s) [upload=6863]: Upload finished. err: <nil> Aug 7 16:16:25.266680963 (6h36m47.010737813s) [upload=6876]: Starting upload Aug 7 16:16:25.266738213 (6h36m47.010795076s) [upload=6876]: Done starting upload Aug 7 16:16:25.266810033 (6h36m47.010866884s) [upload=6876]: Uploading segments... Aug 7 16:16:25.266847650 (6h36m47.010904519s) [upload=6876]: Got next segment. Inline: true Aug 7 16:16:25.266852936 (6h36m47.010909787s) [upload=6876]: Waiting for error group managing segments... Aug 7 16:16:25.266894565 (6h36m47.010951415s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:16:25.266899775 (6h36m47.010956625s) [upload=6876]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:16:25.266903077 (6h36m47.010959927s) [upload=6876]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:16:25.586136069 (6h36m47.330192919s) [upload=6876]: Done uploading segments... Aug 7 16:16:25.586153615 (6h36m47.330210466s) [upload=6876]: Upload finished. err: <nil> Aug 7 16:17:04.757419053 (6h37m26.501475913s) [upload=6888]: Starting upload Aug 7 16:17:04.757525024 (6h37m26.501581888s) [upload=6888]: Done starting upload Aug 7 16:17:04.757628272 (6h37m26.501685132s) [upload=6888]: Uploading segments... Aug 7 16:17:04.757679794 (6h37m26.501736659s) [upload=6888]: Got next segment. Inline: true Aug 7 16:17:04.757684880 (6h37m26.50174174s) [upload=6888]: Waiting for error group managing segments... Aug 7 16:17:04.757726993 (6h37m26.501783856s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:04.757731484 (6h37m26.501788344s) [upload=6888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:04.757734132 (6h37m26.501790992s) [upload=6888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:04.968053417 (6h37m26.712110287s) [upload=6888]: Done uploading segments... Aug 7 16:17:04.968077827 (6h37m26.712134695s) [upload=6888]: Upload finished. err: <nil> Aug 7 16:17:59.760309655 (6h38m21.504366521s) [upload=6904]: Starting upload Aug 7 16:17:59.760463865 (6h38m21.504520712s) [upload=6904]: Done starting upload Aug 7 16:17:59.760588298 (6h38m21.504645149s) [upload=6904]: Uploading segments... Aug 7 16:17:59.760691251 (6h38m21.504748146s) [upload=6904]: Got next segment. Inline: true Aug 7 16:17:59.760700598 (6h38m21.504757713s) [upload=6904]: Waiting for error group managing segments... Aug 7 16:17:59.760761495 (6h38m21.504818358s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:17:59.760768357 (6h38m21.504825219s) [upload=6904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:17:59.760771931 (6h38m21.504828792s) [upload=6904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:17:59.850749254 (6h38m21.594806104s) [upload=6904]: Done uploading segments... Aug 7 16:17:59.850781992 (6h38m21.594838852s) [upload=6904]: Upload finished. err: metaclient: Too Many Requests Duration:* 364.878813ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X...........................X.....X...X.......] duration: 2.737443928s FAIL (2025-08-07 16:17:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.737470831s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X...........................X.....X...X......] duration: 2.537634089s FAIL (2025-08-07 16:16:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.537667165s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X...........................X.....X...X.....] duration: 2.467128435s FAIL (2025-08-07 16:15:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.467158627s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X...........................X.....X...X....] duration: 2.831702318s FAIL (2025-08-07 16:15:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.831807227s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................X...........................X.....X...X...] duration: 2.703532553s FAIL (2025-08-07 16:14:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.703578838s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................X...........................X.....X...X..] duration: 2.847582851s FAIL (2025-08-07 16:13:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.847615144s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................X...........................X.....X...X.] duration: 2.549881092s FAIL (2025-08-07 16:13:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 2.549910957s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................X...........................X.....X...X] duration: 362.632624ms FAIL (2025-08-07 16:12:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Q4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.504660605 (6h31m25.248717471s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505097096 (6h31m25.249153965s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505413078 (6h31m25.249469938s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.505721701 (6h31m25.249778562s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506046555 (6h31m25.250103425s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506120469 (6h31m25.250177337s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506409609 (6h31m25.250466472s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506484858 (6h31m25.250541728s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506748488 (6h31m25.250805359s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.506865146 (6h31m25.250922018s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507142094 (6h31m25.251198978s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507243588 (6h31m25.251300457s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.507515077 (6h31m25.251571943s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508085279 (6h31m25.252142148s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508427396 (6h31m25.252484266s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.508757738 (6h31m25.252814606s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:03.509084661 (6h31m25.253141533s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Uploading piece... Aug 7 16:11:04.078057456 (6h31m25.822114317s) [upload=6775]: Next returned nil segment Aug 7 16:11:04.078073635 (6h31m25.822130485s) [upload=6775]: Waiting for error group managing segments... Aug 7 16:11:04.241516978 (6h31m25.985573832s) [noise=true , address=45.91.116.13:4167 , piece_id=JRM6EKWDUA4Y6I7LG62ZPIZYMVYBW2FHV4EZHN3M2JDMBYUFYZ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249248623 (6h31m25.993305473s) [noise=true , address=104.36.82.62:4439 , piece_id=LXVYYJZCPJ422L6OYYCFMWIHQ6GWSMRY6E7SD4OH2CVK5ZAAS6AA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.249762978 (6h31m25.993819829s) [noise=true , address=104.36.82.14:4245 , piece_id=2E7W74HRM2XJ7HRJIGSYK7GO634KNJNRWFH5FFZ5B7ZKDDDNHMJA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.250519333 (6h31m25.994576195s) [noise=true , address=104.36.82.53:4143 , piece_id=L2TTJ35VO52RMXW6HKWNVJHUGBSDVAC6MRLT7ITGZLDEMNCFISWQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.251309120 (6h31m25.995365971s) [noise=true , address=104.36.82.44:4881 , piece_id=I67JESWFMLPOQ5MMXELRJZ3PEUUPJSECICBXGBKRRW2B734RSAHQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.255147170 (6h31m25.999204021s) [noise=true , address=104.36.82.39:4759 , piece_id=KS6HWWWX2YOBFJ3ICQ76PSFACQ5525Q4573WCG5KBWGG3DUCUUQA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.264155363 (6h31m26.008212213s) [noise=true , address=104.36.82.41:4779 , piece_id=MYUPABYI52KPDBLYPEFYM3OUJCAZ2W35J7EGE5W43ZA6ZJDVDI7Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.267741567 (6h31m26.01179843s) [noise=true , address=104.36.82.60:4347 , piece_id=47NKI5TIYL3VUX6S3TFNL4DKGWOURQEJFPPMBZDEGDNL2K3QXTVQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.268771647 (6h31m26.012828508s) [noise=true , address=104.36.82.19:4426 , piece_id=CFSGZDY434JXRE52ZJNXFB6RSMKTNUGUDOHSXKJLQTCRCMEC3L2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.269393024 (6h31m26.013449872s) [noise=true , address=104.36.82.36:4619 , piece_id=GFQ4VIRJQJJFWBSDTA53BT63ABEIP76DRVO2ALYHNJ3FT543YLCA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.271064839 (6h31m26.01512169s) [noise=true , address=45.91.116.13:4168 , piece_id=OHCQJ6MGLRMQQKVPG7FXYMWD5DVOEK7J5I5LP4AKUYLX4FI2LJ6Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.272275881 (6h31m26.016332731s) [noise=true , address=104.36.82.9:4038 , piece_id=C6HC2WZG2FMSPZ7DAVF5N7NUXAHNZYVE5XT2OLC2VT26MCHQXTGQ, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.275453935 (6h31m26.019510787s) [noise=true , address=104.36.82.59:4334 , piece_id=QBUX5PJTHXJCPUHIKTXRMZQUBVJ52CFAPJ5WMJYXJAJKZP526VDA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277895347 (6h31m26.021952197s) [noise=true , address=104.36.82.17:4333 , piece_id=S6MW3QNVZXXIOPRSG7TXQWGLIPVAOCTMCQCRI72EV6MA4LTKWH5A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: <nil> Aug 7 16:11:04.277937699 (6h31m26.02199456s) [seg_pos={0 0}, upload=6775]: Segment reached optimal threshold of 14 pieces. Aug 7 16:11:04.278113497 (6h31m26.022170348s) [noise=true , address=104.36.82.8:4978 , piece_id=Q52AE3OEHIA4DRUK7BWYKW7RBMQBLHYUOX5E2NLG5YC2K5M2FI2Q, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124urtaxr9hWQSs1Kik6Npue6EaRhaUMjEo8UHJSbz6HWaEFWCX): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278199670 (6h31m26.022256531s) [noise=true , address=104.36.82.23:4467 , piece_id=UNFUPF54IBB6U6A6TWTQSUD72J6QFSNXJSDS3TX5PU3EUQXXVC6A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278218928 (6h31m26.022275775s) [noise=true , address=104.36.82.52:4103 , piece_id=AILC4M4RPDD5PPXC2KZN6YEM4SG2WFMZQE435PPPOHMK752KBJ7A, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu): protocol: expected piece hash; context canceled; EOF Aug 7 16:11:04.278387483 (6h31m26.022444334s) [noise=true , address=64.49.13.139:4476 , piece_id=SSQSIJVXQK2UXKTPBL4QLA257IGBFYMUGJOSPBTBKRE6SOIHUZRA, seg_pos={0 0}, upload=6775]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oXSqFgqNNvDUaEwcHoEVUuaLukC2zaEfosyoeVLbXN81B3vVY): protocol: write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection; write tcp 139.59.144.22:38184->64.49.13.139:4476: use of closed network connection Aug 7 16:11:04.278430839 (6h31m26.022487707s) [seg_pos={0 0}, upload=6775]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:11:04.278578412 (6h31m26.022635282s) [upload=6775]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:11:04.278586114 (6h31m26.022642982s) [upload=6775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:11:04.578500560 (6h31m26.322557451s) [upload=6775]: Done uploading segments... Aug 7 16:11:04.578523238 (6h31m26.322580102s) [upload=6775]: Upload finished. err: <nil> Aug 7 16:12:05.151148886 (6h32m26.895205736s) [upload=6790]: Starting upload Aug 7 16:12:05.151218724 (6h32m26.895275576s) [upload=6790]: Done starting upload Aug 7 16:12:05.151375465 (6h32m26.895432321s) [upload=6790]: Uploading segments... Aug 7 16:12:05.151894056 (6h32m26.895950907s) [upload=6790]: Got next segment. Inline: false Aug 7 16:12:05.151952155 (6h32m26.896009005s) [upload=6790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:12:05.151958015 (6h32m26.896014866s) [upload=6790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:12:05.245599358 (6h32m26.989656224s) [upload=6790]: Next segment err: metaclient: Too Many Requests Aug 7 16:12:05.245626113 (6h32m26.989682975s) [upload=6790]: Done uploading segments... Aug 7 16:12:05.245642458 (6h32m26.989699318s) [upload=6790]: Upload finished. err: metaclient: Too Many Requests Duration: 362.652411ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X...........................X.....X...] duration: 2.533376125s FAIL (2025-08-07 16:11:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== 7]: Starting upload Aug 7 16:08:42.772230857 (6h29m4.516287717s) [upload=6737]: Done starting upload Aug 7 16:08:42.772456498 (6h29m4.516513358s) [upload=6737]: Uploading segments... Aug 7 16:08:42.772538468 (6h29m4.516595319s) [upload=6737]: Got next segment. Inline: false Aug 7 16:08:42.772594534 (6h29m4.516651396s) [upload=6737]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:08:42.772600363 (6h29m4.516657224s) [upload=6737]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:08:43.258342045 (6h29m5.002398905s) [seg_pos={0 0}, upload=6737]: Begin upload segment... Aug 7 16:08:43.258421194 (6h29m5.002478054s) [seg_pos={0 0}, upload=6737]: Done begin upload segment. Aug 7 16:08:43.258476091 (6h29m5.002532957s) [noise=true , address=104.36.82.9:4030 , piece_id=ZE3R673MBZEZHIUSNBWNFO5MKMRISSGQFPK6FUOYYOKXVIMLGE2A, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.258489428 (6h29m5.002546289s) [noise=true , address=104.36.82.37:4676 , piece_id=5TDUZ3FAJGT67RLMDOXHJVUMBZTAWW5GPR4IQ3GDHOTI7TBDDHMQ, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.258926224 (6h29m5.002983088s) [noise=true , address=104.36.82.11:4142 , piece_id=O5KIKCTT44FYJQ43VQGNEBKGDTREYYYTPVQZGOIRSU4VATOV2YRA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.258954738 (6h29m5.003011599s) [noise=true , address=45.91.116.13:4158 , piece_id=24F3IX666JNDGNNACBFV24QDT67OQQ2AVMJ6KXD4MTEZ222IPCMA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.259286682 (6h29m5.003343544s) [noise=true , address=104.36.82.45:4917 , piece_id=H6IKPEZ5RZKG56JJXNK3TS3TVLV4ILNSLUMCGNRGC5DAV4GTWEWA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.259373108 (6h29m5.003429971s) [noise=true , address=104.36.82.3:4793 , piece_id=VXZPLLDPUDDU74RK4XX5F7BJF4TW5A2BNCMKNRJGMFQAVSF2B34Q, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.259758489 (6h29m5.003815356s) [noise=true , address=77.90.31.12:4617 , piece_id=R2XNRXZACNVJY2WOH4Z23EJTEH4ZZSETKWGH2KXABJMKD255BH7Q, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.259958676 (6h29m5.004015536s) [noise=true , address=104.36.82.55:4213 , piece_id=T6UGKVTQC3CZ3SVDXP6FPXONPBJFDKMXQ64JNY6UKKKDEBD2YQOQ, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.260164258 (6h29m5.004221121s) [noise=true , address=64.49.13.140:4510 , piece_id=IJZCVZJ5FW4FHS34YICVWSXFOG7X6GJUX7JEKXJU6W6OJGWQFI3Q, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.260376109 (6h29m5.004432969s) [noise=true , address=104.36.82.18:4366 , piece_id=KLU4T4HQVO57HLSG7QRAEZEOJ446AHAXT762L5DMXFAOCSG3T4HA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.260461484 (6h29m5.004518331s) [noise=true , address=104.36.82.8:4991 , piece_id=WKLOM5AHPRYQ7VPABOUXTLD6BPBFIYMLCOFB6DRWPSH76VS6HQSA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.260791961 (6h29m5.004848827s) [noise=true , address=104.36.82.44:4875 , piece_id=TJS5ZLUXWXPF2K27REYZNSWRWSNX4L6FUB6ML7YCCLAFNKZP7SLQ, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.261199071 (6h29m5.005255934s) [noise=true , address=104.36.82.62:4437 , piece_id=3ZYXCZI4UFGFN3BFYV7MAMDTHJJWNGZHOBL6EJP67MJ6UGGINDVA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.261590392 (6h29m5.005647252s) [noise=true , address=104.36.82.61:4379 , piece_id=CKYZ33XXBF5IYDJICOFSQSJB74KMBYHE7GLB6U5NSVJRWKR3JUUA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.262074480 (6h29m5.00613134s) [noise=true , address=79.127.201.215:4345 , piece_id=QYPARMNAT5HGYN75METGOLGOL46KVVAGWN4I7H7BIUFSPO2ISPOA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.262430036 (6h29m5.006486896s) [noise=true , address=45.91.116.6:4997 , piece_id=NF5NPH2CT5I7AZZHOTWUGI7MEA337Y5EBHD3XPCZLC7HYAKMGRNQ, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.262810216 (6h29m5.006867077s) [noise=true , address=104.36.82.16:4310 , piece_id=LZCDEPEQPTG2UA3S5AXNVEZXP7J7XELJLNZGOCAU54XWMIIS4ZTA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.263218860 (6h29m5.007275723s) [noise=true , address=104.36.82.56:4235 , piece_id=ENC7SHQLFIQKGFKAXUNFMAJE6BRIYAD5MUSFJOAJ27NVA75HC53Q, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.826035186 (6h29m5.570092052s) [upload=6737]: Next returned nil segment Aug 7 16:08:43.826061514 (6h29m5.570118384s) [upload=6737]: Waiting for error group managing segments... Aug 7 16:08:43.982264440 (6h29m5.726321292s) [noise=true , address=77.90.31.12:4617 , piece_id=R2XNRXZACNVJY2WOH4Z23EJTEH4ZZSETKWGH2KXABJMKD255BH7Q, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:43.983701191 (6h29m5.727758053s) [noise=true , address=45.91.116.6:4997 , piece_id=NF5NPH2CT5I7AZZHOTWUGI7MEA337Y5EBHD3XPCZLC7HYAKMGRNQ, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:43.987876406 (6h29m5.731933252s) [noise=true , address=45.91.116.13:4158 , piece_id=24F3IX666JNDGNNACBFV24QDT67OQQ2AVMJ6KXD4MTEZ222IPCMA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.015387030 (6h29m5.759443889s) [noise=true , address=104.36.82.8:4991 , piece_id=WKLOM5AHPRYQ7VPABOUXTLD6BPBFIYMLCOFB6DRWPSH76VS6HQSA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.017659150 (6h29m5.761716003s) [noise=true , address=104.36.82.62:4437 , piece_id=3ZYXCZI4UFGFN3BFYV7MAMDTHJJWNGZHOBL6EJP67MJ6UGGINDVA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.025145909 (6h29m5.769202756s) [noise=true , address=104.36.82.9:4030 , piece_id=ZE3R673MBZEZHIUSNBWNFO5MKMRISSGQFPK6FUOYYOKXVIMLGE2A, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.028503271 (6h29m5.772560119s) [noise=true , address=104.36.82.61:4379 , piece_id=CKYZ33XXBF5IYDJICOFSQSJB74KMBYHE7GLB6U5NSVJRWKR3JUUA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.033358704 (6h29m5.777415553s) [noise=true , address=104.36.82.18:4366 , piece_id=KLU4T4HQVO57HLSG7QRAEZEOJ446AHAXT762L5DMXFAOCSG3T4HA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.033582322 (6h29m5.777639181s) [noise=true , address=104.36.82.55:4213 , piece_id=T6UGKVTQC3CZ3SVDXP6FPXONPBJFDKMXQ64JNY6UKKKDEBD2YQOQ, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.039894908 (6h29m5.783951757s) [noise=true , address=104.36.82.56:4235 , piece_id=ENC7SHQLFIQKGFKAXUNFMAJE6BRIYAD5MUSFJOAJ27NVA75HC53Q, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.041408803 (6h29m5.785465663s) [noise=true , address=104.36.82.11:4142 , piece_id=O5KIKCTT44FYJQ43VQGNEBKGDTREYYYTPVQZGOIRSU4VATOV2YRA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.044287964 (6h29m5.788344824s) [noise=true , address=104.36.82.37:4676 , piece_id=5TDUZ3FAJGT67RLMDOXHJVUMBZTAWW5GPR4IQ3GDHOTI7TBDDHMQ, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.046540253 (6h29m5.790597104s) [noise=true , address=104.36.82.45:4917 , piece_id=H6IKPEZ5RZKG56JJXNK3TS3TVLV4ILNSLUMCGNRGC5DAV4GTWEWA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.096539398 (6h29m5.840596245s) [noise=true , address=104.36.82.16:4310 , piece_id=LZCDEPEQPTG2UA3S5AXNVEZXP7J7XELJLNZGOCAU54XWMIIS4ZTA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.096577560 (6h29m5.84063442s) [seg_pos={0 0}, upload=6737]: Segment reached optimal threshold of 14 pieces. Aug 7 16:08:44.096802002 (6h29m5.840858863s) [noise=true , address=79.127.201.215:4345 , piece_id=QYPARMNAT5HGYN75METGOLGOL46KVVAGWN4I7H7BIUFSPO2ISPOA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u4jspTRDLXgx4VABvYM3XdZ3fnEfvC7rSiAzKKccUy4k3y4F8): protocol: expected piece hash; context canceled; EOF Aug 7 16:08:44.096812578 (6h29m5.84086944s) [noise=true , address=104.36.82.44:4875 , piece_id=TJS5ZLUXWXPF2K27REYZNSWRWSNX4L6FUB6ML7YCCLAFNKZP7SLQ, seg_pos={0 0}, upload=6737]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 7 16:08:44.096903705 (6h29m5.840960566s) [noise=true , address=64.49.13.140:4510 , piece_id=IJZCVZJ5FW4FHS34YICVWSXFOG7X6GJUX7JEKXJU6W6OJGWQFI3Q, seg_pos={0 0}, upload=6737]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: write tcp 139.59.144.22:33862->64.49.13.140:4510: use of closed network connection; write tcp 139.59.144.22:33862->64.49.13.140:4510: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33862->64.49.13.140:4510: use of closed network connection Aug 7 16:08:44.096913388 (6h29m5.84097025s) [noise=true , address=104.36.82.3:4793 , piece_id=VXZPLLDPUDDU74RK4XX5F7BJF4TW5A2BNCMKNRJGMFQAVSF2B34Q, seg_pos={0 0}, upload=6737]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ): protocol: expected piece hash; context canceled; EOF Aug 7 16:08:44.096931125 (6h29m5.840987992s) [seg_pos={0 0}, upload=6737]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:08:44.097034926 (6h29m5.841091791s) [upload=6737]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:08:44.097041122 (6h29m5.841097986s) [upload=6737]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:08:44.314803955 (6h29m6.058860802s) [upload=6737]: Done uploading segments... Aug 7 16:08:44.314832835 (6h29m6.058889683s) [upload=6737]: Upload finished. err: <nil> Duration: 2.533419152s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X...........................X.....X..] duration: 2.634762746s FAIL (2025-08-07 16:10:01 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== 7]: Starting upload Aug 7 16:08:42.772230857 (6h29m4.516287717s) [upload=6737]: Done starting upload Aug 7 16:08:42.772456498 (6h29m4.516513358s) [upload=6737]: Uploading segments... Aug 7 16:08:42.772538468 (6h29m4.516595319s) [upload=6737]: Got next segment. Inline: false Aug 7 16:08:42.772594534 (6h29m4.516651396s) [upload=6737]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:08:42.772600363 (6h29m4.516657224s) [upload=6737]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:08:43.258342045 (6h29m5.002398905s) [seg_pos={0 0}, upload=6737]: Begin upload segment... Aug 7 16:08:43.258421194 (6h29m5.002478054s) [seg_pos={0 0}, upload=6737]: Done begin upload segment. Aug 7 16:08:43.258476091 (6h29m5.002532957s) [noise=true , address=104.36.82.9:4030 , piece_id=ZE3R673MBZEZHIUSNBWNFO5MKMRISSGQFPK6FUOYYOKXVIMLGE2A, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.258489428 (6h29m5.002546289s) [noise=true , address=104.36.82.37:4676 , piece_id=5TDUZ3FAJGT67RLMDOXHJVUMBZTAWW5GPR4IQ3GDHOTI7TBDDHMQ, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.258926224 (6h29m5.002983088s) [noise=true , address=104.36.82.11:4142 , piece_id=O5KIKCTT44FYJQ43VQGNEBKGDTREYYYTPVQZGOIRSU4VATOV2YRA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.258954738 (6h29m5.003011599s) [noise=true , address=45.91.116.13:4158 , piece_id=24F3IX666JNDGNNACBFV24QDT67OQQ2AVMJ6KXD4MTEZ222IPCMA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.259286682 (6h29m5.003343544s) [noise=true , address=104.36.82.45:4917 , piece_id=H6IKPEZ5RZKG56JJXNK3TS3TVLV4ILNSLUMCGNRGC5DAV4GTWEWA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.259373108 (6h29m5.003429971s) [noise=true , address=104.36.82.3:4793 , piece_id=VXZPLLDPUDDU74RK4XX5F7BJF4TW5A2BNCMKNRJGMFQAVSF2B34Q, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.259758489 (6h29m5.003815356s) [noise=true , address=77.90.31.12:4617 , piece_id=R2XNRXZACNVJY2WOH4Z23EJTEH4ZZSETKWGH2KXABJMKD255BH7Q, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.259958676 (6h29m5.004015536s) [noise=true , address=104.36.82.55:4213 , piece_id=T6UGKVTQC3CZ3SVDXP6FPXONPBJFDKMXQ64JNY6UKKKDEBD2YQOQ, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.260164258 (6h29m5.004221121s) [noise=true , address=64.49.13.140:4510 , piece_id=IJZCVZJ5FW4FHS34YICVWSXFOG7X6GJUX7JEKXJU6W6OJGWQFI3Q, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.260376109 (6h29m5.004432969s) [noise=true , address=104.36.82.18:4366 , piece_id=KLU4T4HQVO57HLSG7QRAEZEOJ446AHAXT762L5DMXFAOCSG3T4HA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.260461484 (6h29m5.004518331s) [noise=true , address=104.36.82.8:4991 , piece_id=WKLOM5AHPRYQ7VPABOUXTLD6BPBFIYMLCOFB6DRWPSH76VS6HQSA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.260791961 (6h29m5.004848827s) [noise=true , address=104.36.82.44:4875 , piece_id=TJS5ZLUXWXPF2K27REYZNSWRWSNX4L6FUB6ML7YCCLAFNKZP7SLQ, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.261199071 (6h29m5.005255934s) [noise=true , address=104.36.82.62:4437 , piece_id=3ZYXCZI4UFGFN3BFYV7MAMDTHJJWNGZHOBL6EJP67MJ6UGGINDVA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.261590392 (6h29m5.005647252s) [noise=true , address=104.36.82.61:4379 , piece_id=CKYZ33XXBF5IYDJICOFSQSJB74KMBYHE7GLB6U5NSVJRWKR3JUUA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.262074480 (6h29m5.00613134s) [noise=true , address=79.127.201.215:4345 , piece_id=QYPARMNAT5HGYN75METGOLGOL46KVVAGWN4I7H7BIUFSPO2ISPOA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.262430036 (6h29m5.006486896s) [noise=true , address=45.91.116.6:4997 , piece_id=NF5NPH2CT5I7AZZHOTWUGI7MEA337Y5EBHD3XPCZLC7HYAKMGRNQ, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.262810216 (6h29m5.006867077s) [noise=true , address=104.36.82.16:4310 , piece_id=LZCDEPEQPTG2UA3S5AXNVEZXP7J7XELJLNZGOCAU54XWMIIS4ZTA, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.263218860 (6h29m5.007275723s) [noise=true , address=104.36.82.56:4235 , piece_id=ENC7SHQLFIQKGFKAXUNFMAJE6BRIYAD5MUSFJOAJ27NVA75HC53Q, seg_pos={0 0}, upload=6737]: Uploading piece... Aug 7 16:08:43.826035186 (6h29m5.570092052s) [upload=6737]: Next returned nil segment Aug 7 16:08:43.826061514 (6h29m5.570118384s) [upload=6737]: Waiting for error group managing segments... Aug 7 16:08:43.982264440 (6h29m5.726321292s) [noise=true , address=77.90.31.12:4617 , piece_id=R2XNRXZACNVJY2WOH4Z23EJTEH4ZZSETKWGH2KXABJMKD255BH7Q, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:43.983701191 (6h29m5.727758053s) [noise=true , address=45.91.116.6:4997 , piece_id=NF5NPH2CT5I7AZZHOTWUGI7MEA337Y5EBHD3XPCZLC7HYAKMGRNQ, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:43.987876406 (6h29m5.731933252s) [noise=true , address=45.91.116.13:4158 , piece_id=24F3IX666JNDGNNACBFV24QDT67OQQ2AVMJ6KXD4MTEZ222IPCMA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.015387030 (6h29m5.759443889s) [noise=true , address=104.36.82.8:4991 , piece_id=WKLOM5AHPRYQ7VPABOUXTLD6BPBFIYMLCOFB6DRWPSH76VS6HQSA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.017659150 (6h29m5.761716003s) [noise=true , address=104.36.82.62:4437 , piece_id=3ZYXCZI4UFGFN3BFYV7MAMDTHJJWNGZHOBL6EJP67MJ6UGGINDVA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.025145909 (6h29m5.769202756s) [noise=true , address=104.36.82.9:4030 , piece_id=ZE3R673MBZEZHIUSNBWNFO5MKMRISSGQFPK6FUOYYOKXVIMLGE2A, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.028503271 (6h29m5.772560119s) [noise=true , address=104.36.82.61:4379 , piece_id=CKYZ33XXBF5IYDJICOFSQSJB74KMBYHE7GLB6U5NSVJRWKR3JUUA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.033358704 (6h29m5.777415553s) [noise=true , address=104.36.82.18:4366 , piece_id=KLU4T4HQVO57HLSG7QRAEZEOJ446AHAXT762L5DMXFAOCSG3T4HA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.033582322 (6h29m5.777639181s) [noise=true , address=104.36.82.55:4213 , piece_id=T6UGKVTQC3CZ3SVDXP6FPXONPBJFDKMXQ64JNY6UKKKDEBD2YQOQ, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.039894908 (6h29m5.783951757s) [noise=true , address=104.36.82.56:4235 , piece_id=ENC7SHQLFIQKGFKAXUNFMAJE6BRIYAD5MUSFJOAJ27NVA75HC53Q, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.041408803 (6h29m5.785465663s) [noise=true , address=104.36.82.11:4142 , piece_id=O5KIKCTT44FYJQ43VQGNEBKGDTREYYYTPVQZGOIRSU4VATOV2YRA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.044287964 (6h29m5.788344824s) [noise=true , address=104.36.82.37:4676 , piece_id=5TDUZ3FAJGT67RLMDOXHJVUMBZTAWW5GPR4IQ3GDHOTI7TBDDHMQ, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.046540253 (6h29m5.790597104s) [noise=true , address=104.36.82.45:4917 , piece_id=H6IKPEZ5RZKG56JJXNK3TS3TVLV4ILNSLUMCGNRGC5DAV4GTWEWA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.096539398 (6h29m5.840596245s) [noise=true , address=104.36.82.16:4310 , piece_id=LZCDEPEQPTG2UA3S5AXNVEZXP7J7XELJLNZGOCAU54XWMIIS4ZTA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: <nil> Aug 7 16:08:44.096577560 (6h29m5.84063442s) [seg_pos={0 0}, upload=6737]: Segment reached optimal threshold of 14 pieces. Aug 7 16:08:44.096802002 (6h29m5.840858863s) [noise=true , address=79.127.201.215:4345 , piece_id=QYPARMNAT5HGYN75METGOLGOL46KVVAGWN4I7H7BIUFSPO2ISPOA, seg_pos={0 0}, upload=6737]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u4jspTRDLXgx4VABvYM3XdZ3fnEfvC7rSiAzKKccUy4k3y4F8): protocol: expected piece hash; context canceled; EOF Aug 7 16:08:44.096812578 (6h29m5.84086944s) [noise=true , address=104.36.82.44:4875 , piece_id=TJS5ZLUXWXPF2K27REYZNSWRWSNX4L6FUB6ML7YCCLAFNKZP7SLQ, seg_pos={0 0}, upload=6737]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ijEJgYouyMkUqpLWWz7eH2NGvvs9XpN2ujDjupPykDe5eWcB1): protocol: expected piece hash; context canceled; EOF Aug 7 16:08:44.096903705 (6h29m5.840960566s) [noise=true , address=64.49.13.140:4510 , piece_id=IJZCVZJ5FW4FHS34YICVWSXFOG7X6GJUX7JEKXJU6W6OJGWQFI3Q, seg_pos={0 0}, upload=6737]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hmWsLV5jkkdh53WQaHCK9m7QQY6KBBZtrgZWbcSvWASWYFGJz): protocol: write tcp 139.59.144.22:33862->64.49.13.140:4510: use of closed network connection; write tcp 139.59.144.22:33862->64.49.13.140:4510: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33862->64.49.13.140:4510: use of closed network connection Aug 7 16:08:44.096913388 (6h29m5.84097025s) [noise=true , address=104.36.82.3:4793 , piece_id=VXZPLLDPUDDU74RK4XX5F7BJF4TW5A2BNCMKNRJGMFQAVSF2B34Q, seg_pos={0 0}, upload=6737]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ): protocol: expected piece hash; context canceled; EOF Aug 7 16:08:44.096931125 (6h29m5.840987992s) [seg_pos={0 0}, upload=6737]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:08:44.097034926 (6h29m5.841091791s) [upload=6737]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:08:44.097041122 (6h29m5.841097986s) [upload=6737]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:08:44.314803955 (6h29m6.058860802s) [upload=6737]: Done uploading segments... Aug 7 16:08:44.314832835 (6h29m6.058889683s) [upload=6737]: Upload finished. err: <nil> Duration: 2.634788141s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)