Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.....X..........................X...............X........] duration: 2.570067458s FAIL (2025-08-11 16:30:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.570105499s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 3.180712674s FAIL (2025-08-11 16:30:29 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.18074737s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.....X..........................X...............X.......] duration: 2.645612015s FAIL (2025-08-11 16:30:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.645670433s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 936.946889ms FAIL (2025-08-11 16:30:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 937.063316ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X...X.....X..........................X...............X......] duration: 2.434681915s FAIL (2025-08-11 16:29:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.434737845s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 3.393053584s FAIL (2025-08-11 16:29:33 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.393146445s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X...X.....X..........................X...............X.....] duration: 2.608222229s FAIL (2025-08-11 16:29:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.608255909s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.057277753s FAIL (2025-08-11 16:28:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.057315076s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X...X.....X..........................X...............X....] duration: 2.509853796s FAIL (2025-08-11 16:28:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.509959235s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 3.290811365s FAIL (2025-08-11 16:28:31 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.290840117s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 934.727604ms FAIL (2025-08-11 16:28:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 934.781593ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X...X.....X..........................X...............X...] duration: 2.421062618s FAIL (2025-08-11 16:27:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.421094309s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 3.107082089s FAIL (2025-08-11 16:27:28 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.107111693s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 839.767057ms FAIL (2025-08-11 16:27:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 839.800439ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X...X.....X..........................X...............X..] duration: 2.490775502s FAIL (2025-08-11 16:26:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.490807999s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.649929786s FAIL (2025-08-11 16:26:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.650005384s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 3.664986747s FAIL (2025-08-11 16:26:30 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.665018668s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X...X.....X..........................X...............X.] duration: 2.601606947s FAIL (2025-08-11 16:25:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.601639012s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 879.414697ms FAIL (2025-08-11 16:25:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 879.458552ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 3.243481044s FAIL (2025-08-11 16:25:22 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.243529231s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X...X.....X..........................X...............X] duration: 2.284516525s FAIL (2025-08-11 16:25:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b7fc6319c0e0c6a22f951012665a2657b837d88ecb2798d405d17a334eabc892 != before 07d9f4aaacec0f173a811fefde6054aa1b4323147cc1456f63686ac614c02215 ======LOGS====== Aug 11 16:25:05.511804297 (70h59m27.470901544s) [upload=73978]: Starting upload Aug 11 16:25:05.511893982 (70h59m27.470991245s) [upload=73978]: Done starting upload Aug 11 16:25:05.512013203 (70h59m27.471110456s) [upload=73978]: Uploading segments... Aug 11 16:25:05.512191693 (70h59m27.471288951s) [upload=73978]: Got next segment. Inline: false Aug 11 16:25:05.512266648 (70h59m27.471363882s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:25:05.512274607 (70h59m27.471371857s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:25:05.707692123 (70h59m27.666789355s) [seg_pos={0 0}, upload=73978]: Begin upload segment... Aug 11 16:25:05.707764026 (70h59m27.666861277s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.708202374 (70h59m27.667299629s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710029037 (70h59m27.669126316s) [seg_pos={0 0}, upload=73978]: Done begin upload segment. Aug 11 16:25:05.710092971 (70h59m27.669190228s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.710651020 (70h59m27.669748276s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711095867 (70h59m27.67019312s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.711675800 (70h59m27.670773054s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712097599 (70h59m27.671194849s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712518348 (70h59m27.671615608s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.712925721 (70h59m27.672022976s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713326911 (70h59m27.672424168s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.713702391 (70h59m27.672799646s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714129987 (70h59m27.673227239s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714513653 (70h59m27.67361091s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.714965780 (70h59m27.674063037s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715462259 (70h59m27.674559514s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.715912876 (70h59m27.675010126s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716353053 (70h59m27.675450302s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:05.716789882 (70h59m27.675887137s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Uploading piece... Aug 11 16:25:06.289065107 (70h59m28.248162366s) [upload=73978]: Next returned nil segment Aug 11 16:25:06.289090926 (70h59m28.248188185s) [upload=73978]: Waiting for error group managing segments... Aug 11 16:25:06.441742564 (70h59m28.400839811s) [noise=true , address=104.36.82.38:4708 , piece_id=Q24WSTGO33YQVCCU7C7KBB2OHFH3F7DYHQHALPANCCUZ3ZMJCEHA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.452232012 (70h59m28.411329248s) [noise=true , address=104.36.82.58:4296 , piece_id=U4R2IOXUJKAPLBSV5ZHVBH4WJLC34LT7YUTK2EXV6GBZG3YEE5AQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455573734 (70h59m28.414670987s) [noise=true , address=104.36.82.61:4395 , piece_id=FRAP2RCKKQ4ME2JFX4T33TBWBHCY4XQ2PA4SPFFXJPOFOU2MREZA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.455900316 (70h59m28.414997555s) [noise=true , address=104.36.82.36:4630 , piece_id=D5ZETXXFUCNB2GLX3ID7GPLOGOAMGXA5AVPFMQLYYBVY4EPTWO3A, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.458337246 (70h59m28.417434498s) [noise=true , address=45.91.116.5:4318 , piece_id=HCIRYPXD22VXUP576VTLT3OXFDQIVAO7RKDFJXRMLVOFMJ4LZ4UA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.461373176 (70h59m28.420470423s) [noise=true , address=104.36.82.19:4419 , piece_id=V6T7N6NKDTKBLAQPAVQIGWDOMONPDV46EIYB6EVNLLGFWNTIHDAQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.467882490 (70h59m28.426979737s) [noise=true , address=104.36.82.34:4551 , piece_id=HOIIHPJJ3IQ5VBNKPGNTTM2ZKB63LDXBT325LA3XKFTUULHW546Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.476978590 (70h59m28.43607583s) [noise=true , address=79.127.219.38:4008 , piece_id=FAXCTG4QA2GJFRFOYQ7AEE2BI56UQIFQS6E42DF365EQUPFB2EWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.478530601 (70h59m28.437627842s) [noise=true , address=104.36.82.46:4515 , piece_id=G56WFOZ5HCHPLETBPMUXSHONUFYKD6MQCKJV7KIPSPNUWPUECYBQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.482380114 (70h59m28.441477352s) [noise=true , address=104.36.82.39:4732 , piece_id=H4KMQ4XKCVVIT66SQD5KRQSCALDGFOT54AXTIMBA2WCRBMZMYB6Q, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.484647606 (70h59m28.443744844s) [noise=true , address=104.36.82.16:4320 , piece_id=6C6X473YWK53XH6KT2AMFPNCOVR4DRMFBALPYQH3BJMJA4XMWRUA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.501734931 (70h59m28.46083217s) [noise=true , address=104.36.82.37:4659 , piece_id=DDQP7AAGR737TN6D4IOZ4VZHQZEVEDG3XRNNARF5HC75FK7ZNWXA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.509275105 (70h59m28.468372353s) [noise=true , address=104.36.82.23:4452 , piece_id=J47Q23YQTT735VSKVDNLTJX72XASJFNHULVAKV3LGQI4ER4OSTJA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514816772 (70h59m28.47391401s) [noise=true , address=104.36.82.18:4385 , piece_id=TKPBOXPNAJCOWONVZVBPWPU2SQB73OR7V5AQETX42IZ3EFDG5ZWQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: <nil> Aug 11 16:25:06.514849569 (70h59m28.473946816s) [seg_pos={0 0}, upload=73978]: Segment reached optimal threshold of 14 pieces. Aug 11 16:25:06.515642955 (70h59m28.474740205s) [noise=true , address=79.127.201.215:4354 , piece_id=MRDQ4KWGG3JOI2IYYICLQGTK67D2JBGE54WCSRGANJUVI5HL44KA, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13Jtv8kL5R9oVro2D4ioqsznoJBXyNgJLdj1FcrGN3WWH45rkP): protocol: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38920->79.127.201.215:4354: use of closed network connection Aug 11 16:25:06.515675697 (70h59m28.474772949s) [noise=true , address=64.49.13.27:4968 , piece_id=3NGQCJBP64WES5G7GKJ5WZKSD2JZSPQANEWCDB6YLB66MX7XYYYQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12iMqzmqAR9V6Zxnd3x9pw6HVmcg5UaELNM9nTgGkYiYDxeo1Qk): protocol: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58646->64.49.13.27:4968: use of closed network connection Aug 11 16:25:06.515767382 (70h59m28.474864631s) [noise=true , address=64.49.13.140:4509 , piece_id=3RXN3WZFAZCQORMTBL5BWVQ5CM4C5MXRD5E4T2RA7MZCGSNOKOEQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aAChC1wJq5xDtfBhkBT6UdwoWy5TNwFK3etPKmSkQJr8Fz51P): protocol: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->64.49.13.140:4509: use of closed network connection Aug 11 16:25:06.515824996 (70h59m28.474922242s) [noise=true , address=193.149.164.143:4700 , piece_id=64CESG776DTCS6R2BNAWRCJTRECKRE3BZM5KUUQQR6TUITPHIMXQ, seg_pos={0 0}, upload=73978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tnTEYvqJ6PB2DsuXA28sEuEiGj11StA4qdhcYH1kuoaMkTVdY): protocol: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33938->193.149.164.143:4700: use of closed network connection Aug 11 16:25:06.515845948 (70h59m28.474943201s) [seg_pos={0 0}, upload=73978]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:25:06.516007949 (70h59m28.475105446s) [upload=73978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:25:06.516015725 (70h59m28.475112973s) [upload=73978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:25:06.670522872 (70h59m28.629620134s) [upload=73978]: Done uploading segments... Aug 11 16:25:06.670573351 (70h59m28.629670591s) [upload=73978]: Upload finished. err: <nil> Duration: 2.284555587s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 852.043501ms FAIL (2025-08-11 16:24:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 852.075966ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X...X.....X..........................X...............] duration: 2.565698269s FAIL (2025-08-11 16:24:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.565747558s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 3.201568912s FAIL (2025-08-11 16:24:33 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.201617721s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X...X.....X..........................X..............] duration: 2.380327322s FAIL (2025-08-11 16:24:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.380367686s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 3.24958498s FAIL (2025-08-11 16:24:16 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.249621818s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X...X.....X..........................X.............] duration: 2.935434328s FAIL (2025-08-11 16:23:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.935465814s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.385230697s FAIL (2025-08-11 16:23:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.385259361s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X...X.....X..........................X............] duration: 2.472458535s FAIL (2025-08-11 16:23:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.472491653s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 3.311565936s FAIL (2025-08-11 16:23:26 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.311594988s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 3.352168406s FAIL (2025-08-11 16:23:01 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.352207285s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 864.744968ms FAIL (2025-08-11 16:22:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 864.80034ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X...X.....X..........................X...........] duration: 2.914303441s FAIL (2025-08-11 16:22:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.914333731s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 847.282393ms FAIL (2025-08-11 16:22:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 847.313512ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X...X.....X..........................X..........] duration: 2.488280114s FAIL (2025-08-11 16:21:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.48831511s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 3.177245262s FAIL (2025-08-11 16:21:48 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.177277831s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X...X.....X..........................X.........] duration: 2.484133947s FAIL (2025-08-11 16:21:30 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.484164571s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 3.27939771s FAIL (2025-08-11 16:21:27 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.279430797s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 3.265836414s FAIL (2025-08-11 16:21:08 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.265863975s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 908.291733ms FAIL (2025-08-11 16:21:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 908.405102ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X...X.....X..........................X........] duration: 2.374354574s FAIL (2025-08-11 16:20:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.374385688s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 423.333194ms FAIL (2025-08-11 16:20:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 16:20:18.159018177 (70h54m40.118115432s) [upload=73888]: Starting upload Aug 11 16:20:18.159080072 (70h54m40.118177328s) [upload=73888]: Done starting upload Aug 11 16:20:18.159277818 (70h54m40.118375054s) [upload=73888]: Uploading segments... Aug 11 16:20:18.159338190 (70h54m40.118435443s) [upload=73888]: Got next segment. Inline: true Aug 11 16:20:18.159343910 (70h54m40.118441159s) [upload=73888]: Waiting for error group managing segments... Aug 11 16:20:18.159389878 (70h54m40.118487123s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:18.159395133 (70h54m40.118492378s) [upload=73888]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:20:18.159398696 (70h54m40.118495943s) [upload=73888]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:18.296493894 (70h54m40.255591129s) [upload=73888]: Done uploading segments... Aug 11 16:20:18.296513582 (70h54m40.255610822s) [upload=73888]: Upload finished. err: metaclient: Too Many Requests Duration:* 423.371968ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 3.358712348s FAIL (2025-08-11 16:20:01 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.35880114s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X...X.....X..........................X.......] duration: 2.944973906s FAIL (2025-08-11 16:19:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.944996284s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 902.590365ms FAIL (2025-08-11 16:19:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 902.646449ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 3.351251921s FAIL (2025-08-11 16:19:16 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.351287049s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X...X.....X..........................X......] duration: 2.367106286s FAIL (2025-08-11 16:18:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.367137355s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 3.472113771s FAIL (2025-08-11 16:18:29 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.472149223s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 878.577567ms FAIL (2025-08-11 16:18:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 878.614226ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 3.422937387s FAIL (2025-08-11 16:17:49 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.422971489s core uplink ap1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)