Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X......................................X..X] duration: 368.025765ms FAIL (2025-08-19 07:25:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:25:19.855695330 (253h59m41.814792562s) [upload=264402]: Starting upload Aug 19 07:25:19.855746609 (253h59m41.814843854s) [upload=264402]: Done starting upload Aug 19 07:25:19.855829551 (253h59m41.814926804s) [upload=264402]: Uploading segments... Aug 19 07:25:19.856140224 (253h59m41.815237472s) [upload=264402]: Got next segment. Inline: false Aug 19 07:25:19.856189626 (253h59m41.815286875s) [upload=264402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:25:19.856195691 (253h59m41.815292938s) [upload=264402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 07:25:19.946205701 (253h59m41.905302953s) [upload=264402]: Next segment err: metaclient: Too Many Requests Aug 19 07:25:19.946245590 (253h59m41.905342838s) [upload=264402]: Done uploading segments... Aug 19 07:25:19.946280696 (253h59m41.905377927s) [upload=264402]: Upload finished. err: metaclient: Too Many Requests Duration: 368.057914ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.057073886s FAIL (2025-08-19 07:25:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.057106898s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X......................................X..] duration: 2.381422215s FAIL (2025-08-19 07:24:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after dd9a7659defb788177b2f901896af284e919bdea8720708bb14c5364a56e5321 != before 88c8db646f6a377b7a0d8e9ea1fba3fd92c3cf063593b4cc706efbd51b54d3ee ======LOGS====== Aug 19 07:22:27.306173033 (253h56m49.265270268s) [upload=264353]: Starting upload Aug 19 07:22:27.306231214 (253h56m49.265328451s) [upload=264353]: Done starting upload Aug 19 07:22:27.306386984 (253h56m49.26548422s) [upload=264353]: Uploading segments... Aug 19 07:22:27.306557574 (253h56m49.265654812s) [upload=264353]: Got next segment. Inline: false Aug 19 07:22:27.306595267 (253h56m49.265692513s) [upload=264353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:22:27.306601177 (253h56m49.265698422s) [upload=264353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 07:22:27.500693995 (253h56m49.459791242s) [seg_pos={0 0}, upload=264353]: Begin upload segment... Aug 19 07:22:27.500769095 (253h56m49.459866332s) [seg_pos={0 0}, upload=264353]: Done begin upload segment. Aug 19 07:22:27.500853983 (253h56m49.459951233s) [noise=true , address=77.90.31.15:4714 , piece_id=WHITW3MM22QUDV24SYXS6AOBNZRI7YVYWVVKMS22BXQIJ3A74RHQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.500912748 (253h56m49.460010021s) [noise=true , address=79.127.219.47:4062 , piece_id=7GPMR3NQNUHMIBRV3ITEFDXYB7CNJ3CBYJ4OCIJ6U4QWTNCQ4I6Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501472234 (253h56m49.46056949s) [noise=true , address=77.90.31.16:4752 , piece_id=ICJWERASGL5LT4CGOXV6ZGBAWAGEZTX4HGKR4KLEWTIV232D6UFA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501533271 (253h56m49.46063052s) [noise=true , address=45.91.116.11:4126 , piece_id=CG7PWNU5S5WMVNG5NRKR6WVO6BO26WOMWE4UTSL7I3MDHKKU3Q2Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501876767 (253h56m49.460974029s) [noise=true , address=45.91.116.4:4942 , piece_id=Y3IWIYOU35MMEU2PVEFO25FYO5QHSSYKB37VYC6DBTHWUR7TAFRA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502039657 (253h56m49.461136907s) [noise=true , address=79.127.219.48:4099 , piece_id=7DILEI2R4N6QSCF54VTPIRUUQVSPQHEDZUCXC7AQCEIIPTOIK33Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502305913 (253h56m49.461403169s) [noise=true , address=79.127.219.38:4027 , piece_id=DH4RWYLF5YV3XFEQINWKWTZJLSZHBLB7ACTIYWQ5WKK3DOQH376Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502603086 (253h56m49.461700333s) [noise=true , address=104.36.82.44:4880 , piece_id=ERYECQX26N2MKLPNOAGMVWBTLQBOJQUGX26QBHXJVUP6PUPPUM7A, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502736879 (253h56m49.461834136s) [noise=true , address=45.91.116.9:4286 , piece_id=OWWETSANT2N77SO2RQF3XECSTXATGP45UTMTVBJPJL3YGFAJJXAQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503152486 (253h56m49.462249732s) [noise=true , address=77.90.31.10:4538 , piece_id=JVTJFJBGFTU6Y3GKCCVQOCRB4TSF6IPPL4ACPC7KS7B3UWYJVWVQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503190929 (253h56m49.462288185s) [noise=true , address=77.90.31.17:4792 , piece_id=2IFN33MGBTEOVF7F5IDZCGYJFLL3UXT3YTOK43SCWNAWNUWGJY5A, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503625981 (253h56m49.462723237s) [noise=true , address=45.91.116.14:4223 , piece_id=BWUBRSQIVCKKFEQR3KP6ML6Y3HC33NGXXFRCP5INO3UYNX375NHQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503736363 (253h56m49.462833613s) [noise=true , address=104.36.82.60:4372 , piece_id=PR4H5TOHVFIP7DYLGHXI5FMSSSERZR4VBNVL577PZ4RPRKWGBPLA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.504058695 (253h56m49.46315595s) [noise=true , address=45.91.116.8:4076 , piece_id=F44MTWGUDG5L3ZKAWUNE3H6FWT77HAFQTNLBB6CZVJ4B63R3RQHA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.504939023 (253h56m49.464036272s) [noise=true , address=77.90.31.13:4631 , piece_id=WYKANSN542ACDRKAG34Q2QHIH2745W5IVRY63BFTPXRF4VCNJ3ZA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.506678218 (253h56m49.465775469s) [noise=true , address=79.127.219.38:4025 , piece_id=45JFGGCIS4KGNVGX5AQESOEDTMIRZMYQLFQUWOSR2UW3INL5UKUQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.508301162 (253h56m49.467398419s) [noise=true , address=77.90.31.18:4833 , piece_id=222BOUPZJNNW3JKP54VMTLGI4LJZJJ6ULDM6Y2CG2DO63CTGAFAA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.508757969 (253h56m49.467855222s) [noise=true , address=104.36.82.44:4883 , piece_id=CMZ5TFGCY7AQHCKPAPU6YTBGKWW3PXZAWKR5LVYJLYDZCIPMKPPQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:28.074611319 (253h56m50.033708553s) [upload=264353]: Next returned nil segment Aug 19 07:22:28.074631679 (253h56m50.033728926s) [upload=264353]: Waiting for error group managing segments... Aug 19 07:22:28.214932712 (253h56m50.174029965s) [noise=true , address=77.90.31.18:4833 , piece_id=222BOUPZJNNW3JKP54VMTLGI4LJZJJ6ULDM6Y2CG2DO63CTGAFAA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.215334405 (253h56m50.174431641s) [noise=true , address=45.91.116.9:4286 , piece_id=OWWETSANT2N77SO2RQF3XECSTXATGP45UTMTVBJPJL3YGFAJJXAQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.225872394 (253h56m50.184969643s) [noise=true , address=45.91.116.11:4126 , piece_id=CG7PWNU5S5WMVNG5NRKR6WVO6BO26WOMWE4UTSL7I3MDHKKU3Q2Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.233664340 (253h56m50.192761586s) [noise=true , address=104.36.82.44:4880 , piece_id=ERYECQX26N2MKLPNOAGMVWBTLQBOJQUGX26QBHXJVUP6PUPPUM7A, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.236177411 (253h56m50.195274657s) [noise=true , address=77.90.31.10:4538 , piece_id=JVTJFJBGFTU6Y3GKCCVQOCRB4TSF6IPPL4ACPC7KS7B3UWYJVWVQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.238590951 (253h56m50.197688196s) [noise=true , address=79.127.219.38:4025 , piece_id=45JFGGCIS4KGNVGX5AQESOEDTMIRZMYQLFQUWOSR2UW3INL5UKUQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.240505639 (253h56m50.199602885s) [noise=true , address=45.91.116.14:4223 , piece_id=BWUBRSQIVCKKFEQR3KP6ML6Y3HC33NGXXFRCP5INO3UYNX375NHQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.244467665 (253h56m50.203564916s) [noise=true , address=45.91.116.4:4942 , piece_id=Y3IWIYOU35MMEU2PVEFO25FYO5QHSSYKB37VYC6DBTHWUR7TAFRA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.250301434 (253h56m50.209398681s) [noise=true , address=77.90.31.17:4792 , piece_id=2IFN33MGBTEOVF7F5IDZCGYJFLL3UXT3YTOK43SCWNAWNUWGJY5A, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.255974340 (253h56m50.215071592s) [noise=true , address=45.91.116.8:4076 , piece_id=F44MTWGUDG5L3ZKAWUNE3H6FWT77HAFQTNLBB6CZVJ4B63R3RQHA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.256353715 (253h56m50.215450953s) [noise=true , address=104.36.82.44:4883 , piece_id=CMZ5TFGCY7AQHCKPAPU6YTBGKWW3PXZAWKR5LVYJLYDZCIPMKPPQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.267076812 (253h56m50.226174049s) [noise=true , address=79.127.219.38:4027 , piece_id=DH4RWYLF5YV3XFEQINWKWTZJLSZHBLB7ACTIYWQ5WKK3DOQH376Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.271613023 (253h56m50.230710256s) [noise=true , address=79.127.219.48:4099 , piece_id=7DILEI2R4N6QSCF54VTPIRUUQVSPQHEDZUCXC7AQCEIIPTOIK33Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.282639911 (253h56m50.241737162s) [noise=true , address=77.90.31.16:4752 , piece_id=ICJWERASGL5LT4CGOXV6ZGBAWAGEZTX4HGKR4KLEWTIV232D6UFA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.282686495 (253h56m50.241784014s) [seg_pos={0 0}, upload=264353]: Segment reached optimal threshold of 14 pieces. Aug 19 07:22:28.282887308 (253h56m50.241984556s) [noise=true , address=77.90.31.15:4714 , piece_id=WHITW3MM22QUDV24SYXS6AOBNZRI7YVYWVVKMS22BXQIJ3A74RHQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.282948549 (253h56m50.242045808s) [noise=true , address=104.36.82.60:4372 , piece_id=PR4H5TOHVFIP7DYLGHXI5FMSSSERZR4VBNVL577PZ4RPRKWGBPLA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.282990354 (253h56m50.242087601s) [noise=true , address=79.127.219.47:4062 , piece_id=7GPMR3NQNUHMIBRV3ITEFDXYB7CNJ3CBYJ4OCIJ6U4QWTNCQ4I6Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWXxGocse7Wur1vwWUY4e832MJq1L7BYBf4psNP6RpJTSJuMB): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.283071625 (253h56m50.242168871s) [noise=true , address=77.90.31.13:4631 , piece_id=WYKANSN542ACDRKAG34Q2QHIH2745W5IVRY63BFTPXRF4VCNJ3ZA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aB5TGJszYj5bvk2cDbiQA7dcAoTxeYMsYgMWkkn1tACCxdtpz): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.283094976 (253h56m50.242192221s) [seg_pos={0 0}, upload=264353]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 07:22:28.283189406 (253h56m50.242286641s) [upload=264353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 07:22:28.283198033 (253h56m50.242295278s) [upload=264353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:22:28.426563808 (253h56m50.385661094s) [upload=264353]: Done uploading segments... Aug 19 07:22:28.426606287 (253h56m50.385703533s) [upload=264353]: Upload finished. err: <nil> Duration: 2.381457645s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.128306516s FAIL (2025-08-19 07:24:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.128346931s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X......................................X.] duration: 2.435799351s FAIL (2025-08-19 07:23:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after dd9a7659defb788177b2f901896af284e919bdea8720708bb14c5364a56e5321 != before 88c8db646f6a377b7a0d8e9ea1fba3fd92c3cf063593b4cc706efbd51b54d3ee ======LOGS====== Aug 19 07:22:27.306173033 (253h56m49.265270268s) [upload=264353]: Starting upload Aug 19 07:22:27.306231214 (253h56m49.265328451s) [upload=264353]: Done starting upload Aug 19 07:22:27.306386984 (253h56m49.26548422s) [upload=264353]: Uploading segments... Aug 19 07:22:27.306557574 (253h56m49.265654812s) [upload=264353]: Got next segment. Inline: false Aug 19 07:22:27.306595267 (253h56m49.265692513s) [upload=264353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:22:27.306601177 (253h56m49.265698422s) [upload=264353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 07:22:27.500693995 (253h56m49.459791242s) [seg_pos={0 0}, upload=264353]: Begin upload segment... Aug 19 07:22:27.500769095 (253h56m49.459866332s) [seg_pos={0 0}, upload=264353]: Done begin upload segment. Aug 19 07:22:27.500853983 (253h56m49.459951233s) [noise=true , address=77.90.31.15:4714 , piece_id=WHITW3MM22QUDV24SYXS6AOBNZRI7YVYWVVKMS22BXQIJ3A74RHQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.500912748 (253h56m49.460010021s) [noise=true , address=79.127.219.47:4062 , piece_id=7GPMR3NQNUHMIBRV3ITEFDXYB7CNJ3CBYJ4OCIJ6U4QWTNCQ4I6Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501472234 (253h56m49.46056949s) [noise=true , address=77.90.31.16:4752 , piece_id=ICJWERASGL5LT4CGOXV6ZGBAWAGEZTX4HGKR4KLEWTIV232D6UFA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501533271 (253h56m49.46063052s) [noise=true , address=45.91.116.11:4126 , piece_id=CG7PWNU5S5WMVNG5NRKR6WVO6BO26WOMWE4UTSL7I3MDHKKU3Q2Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501876767 (253h56m49.460974029s) [noise=true , address=45.91.116.4:4942 , piece_id=Y3IWIYOU35MMEU2PVEFO25FYO5QHSSYKB37VYC6DBTHWUR7TAFRA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502039657 (253h56m49.461136907s) [noise=true , address=79.127.219.48:4099 , piece_id=7DILEI2R4N6QSCF54VTPIRUUQVSPQHEDZUCXC7AQCEIIPTOIK33Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502305913 (253h56m49.461403169s) [noise=true , address=79.127.219.38:4027 , piece_id=DH4RWYLF5YV3XFEQINWKWTZJLSZHBLB7ACTIYWQ5WKK3DOQH376Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502603086 (253h56m49.461700333s) [noise=true , address=104.36.82.44:4880 , piece_id=ERYECQX26N2MKLPNOAGMVWBTLQBOJQUGX26QBHXJVUP6PUPPUM7A, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502736879 (253h56m49.461834136s) [noise=true , address=45.91.116.9:4286 , piece_id=OWWETSANT2N77SO2RQF3XECSTXATGP45UTMTVBJPJL3YGFAJJXAQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503152486 (253h56m49.462249732s) [noise=true , address=77.90.31.10:4538 , piece_id=JVTJFJBGFTU6Y3GKCCVQOCRB4TSF6IPPL4ACPC7KS7B3UWYJVWVQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503190929 (253h56m49.462288185s) [noise=true , address=77.90.31.17:4792 , piece_id=2IFN33MGBTEOVF7F5IDZCGYJFLL3UXT3YTOK43SCWNAWNUWGJY5A, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503625981 (253h56m49.462723237s) [noise=true , address=45.91.116.14:4223 , piece_id=BWUBRSQIVCKKFEQR3KP6ML6Y3HC33NGXXFRCP5INO3UYNX375NHQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503736363 (253h56m49.462833613s) [noise=true , address=104.36.82.60:4372 , piece_id=PR4H5TOHVFIP7DYLGHXI5FMSSSERZR4VBNVL577PZ4RPRKWGBPLA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.504058695 (253h56m49.46315595s) [noise=true , address=45.91.116.8:4076 , piece_id=F44MTWGUDG5L3ZKAWUNE3H6FWT77HAFQTNLBB6CZVJ4B63R3RQHA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.504939023 (253h56m49.464036272s) [noise=true , address=77.90.31.13:4631 , piece_id=WYKANSN542ACDRKAG34Q2QHIH2745W5IVRY63BFTPXRF4VCNJ3ZA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.506678218 (253h56m49.465775469s) [noise=true , address=79.127.219.38:4025 , piece_id=45JFGGCIS4KGNVGX5AQESOEDTMIRZMYQLFQUWOSR2UW3INL5UKUQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.508301162 (253h56m49.467398419s) [noise=true , address=77.90.31.18:4833 , piece_id=222BOUPZJNNW3JKP54VMTLGI4LJZJJ6ULDM6Y2CG2DO63CTGAFAA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.508757969 (253h56m49.467855222s) [noise=true , address=104.36.82.44:4883 , piece_id=CMZ5TFGCY7AQHCKPAPU6YTBGKWW3PXZAWKR5LVYJLYDZCIPMKPPQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:28.074611319 (253h56m50.033708553s) [upload=264353]: Next returned nil segment Aug 19 07:22:28.074631679 (253h56m50.033728926s) [upload=264353]: Waiting for error group managing segments... Aug 19 07:22:28.214932712 (253h56m50.174029965s) [noise=true , address=77.90.31.18:4833 , piece_id=222BOUPZJNNW3JKP54VMTLGI4LJZJJ6ULDM6Y2CG2DO63CTGAFAA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.215334405 (253h56m50.174431641s) [noise=true , address=45.91.116.9:4286 , piece_id=OWWETSANT2N77SO2RQF3XECSTXATGP45UTMTVBJPJL3YGFAJJXAQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.225872394 (253h56m50.184969643s) [noise=true , address=45.91.116.11:4126 , piece_id=CG7PWNU5S5WMVNG5NRKR6WVO6BO26WOMWE4UTSL7I3MDHKKU3Q2Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.233664340 (253h56m50.192761586s) [noise=true , address=104.36.82.44:4880 , piece_id=ERYECQX26N2MKLPNOAGMVWBTLQBOJQUGX26QBHXJVUP6PUPPUM7A, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.236177411 (253h56m50.195274657s) [noise=true , address=77.90.31.10:4538 , piece_id=JVTJFJBGFTU6Y3GKCCVQOCRB4TSF6IPPL4ACPC7KS7B3UWYJVWVQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.238590951 (253h56m50.197688196s) [noise=true , address=79.127.219.38:4025 , piece_id=45JFGGCIS4KGNVGX5AQESOEDTMIRZMYQLFQUWOSR2UW3INL5UKUQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.240505639 (253h56m50.199602885s) [noise=true , address=45.91.116.14:4223 , piece_id=BWUBRSQIVCKKFEQR3KP6ML6Y3HC33NGXXFRCP5INO3UYNX375NHQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.244467665 (253h56m50.203564916s) [noise=true , address=45.91.116.4:4942 , piece_id=Y3IWIYOU35MMEU2PVEFO25FYO5QHSSYKB37VYC6DBTHWUR7TAFRA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.250301434 (253h56m50.209398681s) [noise=true , address=77.90.31.17:4792 , piece_id=2IFN33MGBTEOVF7F5IDZCGYJFLL3UXT3YTOK43SCWNAWNUWGJY5A, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.255974340 (253h56m50.215071592s) [noise=true , address=45.91.116.8:4076 , piece_id=F44MTWGUDG5L3ZKAWUNE3H6FWT77HAFQTNLBB6CZVJ4B63R3RQHA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.256353715 (253h56m50.215450953s) [noise=true , address=104.36.82.44:4883 , piece_id=CMZ5TFGCY7AQHCKPAPU6YTBGKWW3PXZAWKR5LVYJLYDZCIPMKPPQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.267076812 (253h56m50.226174049s) [noise=true , address=79.127.219.38:4027 , piece_id=DH4RWYLF5YV3XFEQINWKWTZJLSZHBLB7ACTIYWQ5WKK3DOQH376Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.271613023 (253h56m50.230710256s) [noise=true , address=79.127.219.48:4099 , piece_id=7DILEI2R4N6QSCF54VTPIRUUQVSPQHEDZUCXC7AQCEIIPTOIK33Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.282639911 (253h56m50.241737162s) [noise=true , address=77.90.31.16:4752 , piece_id=ICJWERASGL5LT4CGOXV6ZGBAWAGEZTX4HGKR4KLEWTIV232D6UFA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.282686495 (253h56m50.241784014s) [seg_pos={0 0}, upload=264353]: Segment reached optimal threshold of 14 pieces. Aug 19 07:22:28.282887308 (253h56m50.241984556s) [noise=true , address=77.90.31.15:4714 , piece_id=WHITW3MM22QUDV24SYXS6AOBNZRI7YVYWVVKMS22BXQIJ3A74RHQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.282948549 (253h56m50.242045808s) [noise=true , address=104.36.82.60:4372 , piece_id=PR4H5TOHVFIP7DYLGHXI5FMSSSERZR4VBNVL577PZ4RPRKWGBPLA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.282990354 (253h56m50.242087601s) [noise=true , address=79.127.219.47:4062 , piece_id=7GPMR3NQNUHMIBRV3ITEFDXYB7CNJ3CBYJ4OCIJ6U4QWTNCQ4I6Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWXxGocse7Wur1vwWUY4e832MJq1L7BYBf4psNP6RpJTSJuMB): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.283071625 (253h56m50.242168871s) [noise=true , address=77.90.31.13:4631 , piece_id=WYKANSN542ACDRKAG34Q2QHIH2745W5IVRY63BFTPXRF4VCNJ3ZA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aB5TGJszYj5bvk2cDbiQA7dcAoTxeYMsYgMWkkn1tACCxdtpz): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.283094976 (253h56m50.242192221s) [seg_pos={0 0}, upload=264353]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 07:22:28.283189406 (253h56m50.242286641s) [upload=264353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 07:22:28.283198033 (253h56m50.242295278s) [upload=264353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:22:28.426563808 (253h56m50.385661094s) [upload=264353]: Done uploading segments... Aug 19 07:22:28.426606287 (253h56m50.385703533s) [upload=264353]: Upload finished. err: <nil> Duration: 2.435836607s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 852.651266ms FAIL (2025-08-19 07:23:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 852.699631ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.044477687s FAIL (2025-08-19 07:23:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.044511587s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 955.134019ms FAIL (2025-08-19 07:22:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 955.184309ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X......................................X] duration: 2.326980761s FAIL (2025-08-19 07:22:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after dd9a7659defb788177b2f901896af284e919bdea8720708bb14c5364a56e5321 != before 88c8db646f6a377b7a0d8e9ea1fba3fd92c3cf063593b4cc706efbd51b54d3ee ======LOGS====== Aug 19 07:22:27.306173033 (253h56m49.265270268s) [upload=264353]: Starting upload Aug 19 07:22:27.306231214 (253h56m49.265328451s) [upload=264353]: Done starting upload Aug 19 07:22:27.306386984 (253h56m49.26548422s) [upload=264353]: Uploading segments... Aug 19 07:22:27.306557574 (253h56m49.265654812s) [upload=264353]: Got next segment. Inline: false Aug 19 07:22:27.306595267 (253h56m49.265692513s) [upload=264353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:22:27.306601177 (253h56m49.265698422s) [upload=264353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 07:22:27.500693995 (253h56m49.459791242s) [seg_pos={0 0}, upload=264353]: Begin upload segment... Aug 19 07:22:27.500769095 (253h56m49.459866332s) [seg_pos={0 0}, upload=264353]: Done begin upload segment. Aug 19 07:22:27.500853983 (253h56m49.459951233s) [noise=true , address=77.90.31.15:4714 , piece_id=WHITW3MM22QUDV24SYXS6AOBNZRI7YVYWVVKMS22BXQIJ3A74RHQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.500912748 (253h56m49.460010021s) [noise=true , address=79.127.219.47:4062 , piece_id=7GPMR3NQNUHMIBRV3ITEFDXYB7CNJ3CBYJ4OCIJ6U4QWTNCQ4I6Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501472234 (253h56m49.46056949s) [noise=true , address=77.90.31.16:4752 , piece_id=ICJWERASGL5LT4CGOXV6ZGBAWAGEZTX4HGKR4KLEWTIV232D6UFA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501533271 (253h56m49.46063052s) [noise=true , address=45.91.116.11:4126 , piece_id=CG7PWNU5S5WMVNG5NRKR6WVO6BO26WOMWE4UTSL7I3MDHKKU3Q2Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.501876767 (253h56m49.460974029s) [noise=true , address=45.91.116.4:4942 , piece_id=Y3IWIYOU35MMEU2PVEFO25FYO5QHSSYKB37VYC6DBTHWUR7TAFRA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502039657 (253h56m49.461136907s) [noise=true , address=79.127.219.48:4099 , piece_id=7DILEI2R4N6QSCF54VTPIRUUQVSPQHEDZUCXC7AQCEIIPTOIK33Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502305913 (253h56m49.461403169s) [noise=true , address=79.127.219.38:4027 , piece_id=DH4RWYLF5YV3XFEQINWKWTZJLSZHBLB7ACTIYWQ5WKK3DOQH376Q, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502603086 (253h56m49.461700333s) [noise=true , address=104.36.82.44:4880 , piece_id=ERYECQX26N2MKLPNOAGMVWBTLQBOJQUGX26QBHXJVUP6PUPPUM7A, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.502736879 (253h56m49.461834136s) [noise=true , address=45.91.116.9:4286 , piece_id=OWWETSANT2N77SO2RQF3XECSTXATGP45UTMTVBJPJL3YGFAJJXAQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503152486 (253h56m49.462249732s) [noise=true , address=77.90.31.10:4538 , piece_id=JVTJFJBGFTU6Y3GKCCVQOCRB4TSF6IPPL4ACPC7KS7B3UWYJVWVQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503190929 (253h56m49.462288185s) [noise=true , address=77.90.31.17:4792 , piece_id=2IFN33MGBTEOVF7F5IDZCGYJFLL3UXT3YTOK43SCWNAWNUWGJY5A, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503625981 (253h56m49.462723237s) [noise=true , address=45.91.116.14:4223 , piece_id=BWUBRSQIVCKKFEQR3KP6ML6Y3HC33NGXXFRCP5INO3UYNX375NHQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.503736363 (253h56m49.462833613s) [noise=true , address=104.36.82.60:4372 , piece_id=PR4H5TOHVFIP7DYLGHXI5FMSSSERZR4VBNVL577PZ4RPRKWGBPLA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.504058695 (253h56m49.46315595s) [noise=true , address=45.91.116.8:4076 , piece_id=F44MTWGUDG5L3ZKAWUNE3H6FWT77HAFQTNLBB6CZVJ4B63R3RQHA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.504939023 (253h56m49.464036272s) [noise=true , address=77.90.31.13:4631 , piece_id=WYKANSN542ACDRKAG34Q2QHIH2745W5IVRY63BFTPXRF4VCNJ3ZA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.506678218 (253h56m49.465775469s) [noise=true , address=79.127.219.38:4025 , piece_id=45JFGGCIS4KGNVGX5AQESOEDTMIRZMYQLFQUWOSR2UW3INL5UKUQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.508301162 (253h56m49.467398419s) [noise=true , address=77.90.31.18:4833 , piece_id=222BOUPZJNNW3JKP54VMTLGI4LJZJJ6ULDM6Y2CG2DO63CTGAFAA, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:27.508757969 (253h56m49.467855222s) [noise=true , address=104.36.82.44:4883 , piece_id=CMZ5TFGCY7AQHCKPAPU6YTBGKWW3PXZAWKR5LVYJLYDZCIPMKPPQ, seg_pos={0 0}, upload=264353]: Uploading piece... Aug 19 07:22:28.074611319 (253h56m50.033708553s) [upload=264353]: Next returned nil segment Aug 19 07:22:28.074631679 (253h56m50.033728926s) [upload=264353]: Waiting for error group managing segments... Aug 19 07:22:28.214932712 (253h56m50.174029965s) [noise=true , address=77.90.31.18:4833 , piece_id=222BOUPZJNNW3JKP54VMTLGI4LJZJJ6ULDM6Y2CG2DO63CTGAFAA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.215334405 (253h56m50.174431641s) [noise=true , address=45.91.116.9:4286 , piece_id=OWWETSANT2N77SO2RQF3XECSTXATGP45UTMTVBJPJL3YGFAJJXAQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.225872394 (253h56m50.184969643s) [noise=true , address=45.91.116.11:4126 , piece_id=CG7PWNU5S5WMVNG5NRKR6WVO6BO26WOMWE4UTSL7I3MDHKKU3Q2Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.233664340 (253h56m50.192761586s) [noise=true , address=104.36.82.44:4880 , piece_id=ERYECQX26N2MKLPNOAGMVWBTLQBOJQUGX26QBHXJVUP6PUPPUM7A, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.236177411 (253h56m50.195274657s) [noise=true , address=77.90.31.10:4538 , piece_id=JVTJFJBGFTU6Y3GKCCVQOCRB4TSF6IPPL4ACPC7KS7B3UWYJVWVQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.238590951 (253h56m50.197688196s) [noise=true , address=79.127.219.38:4025 , piece_id=45JFGGCIS4KGNVGX5AQESOEDTMIRZMYQLFQUWOSR2UW3INL5UKUQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.240505639 (253h56m50.199602885s) [noise=true , address=45.91.116.14:4223 , piece_id=BWUBRSQIVCKKFEQR3KP6ML6Y3HC33NGXXFRCP5INO3UYNX375NHQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.244467665 (253h56m50.203564916s) [noise=true , address=45.91.116.4:4942 , piece_id=Y3IWIYOU35MMEU2PVEFO25FYO5QHSSYKB37VYC6DBTHWUR7TAFRA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.250301434 (253h56m50.209398681s) [noise=true , address=77.90.31.17:4792 , piece_id=2IFN33MGBTEOVF7F5IDZCGYJFLL3UXT3YTOK43SCWNAWNUWGJY5A, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.255974340 (253h56m50.215071592s) [noise=true , address=45.91.116.8:4076 , piece_id=F44MTWGUDG5L3ZKAWUNE3H6FWT77HAFQTNLBB6CZVJ4B63R3RQHA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.256353715 (253h56m50.215450953s) [noise=true , address=104.36.82.44:4883 , piece_id=CMZ5TFGCY7AQHCKPAPU6YTBGKWW3PXZAWKR5LVYJLYDZCIPMKPPQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.267076812 (253h56m50.226174049s) [noise=true , address=79.127.219.38:4027 , piece_id=DH4RWYLF5YV3XFEQINWKWTZJLSZHBLB7ACTIYWQ5WKK3DOQH376Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.271613023 (253h56m50.230710256s) [noise=true , address=79.127.219.48:4099 , piece_id=7DILEI2R4N6QSCF54VTPIRUUQVSPQHEDZUCXC7AQCEIIPTOIK33Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.282639911 (253h56m50.241737162s) [noise=true , address=77.90.31.16:4752 , piece_id=ICJWERASGL5LT4CGOXV6ZGBAWAGEZTX4HGKR4KLEWTIV232D6UFA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: <nil> Aug 19 07:22:28.282686495 (253h56m50.241784014s) [seg_pos={0 0}, upload=264353]: Segment reached optimal threshold of 14 pieces. Aug 19 07:22:28.282887308 (253h56m50.241984556s) [noise=true , address=77.90.31.15:4714 , piece_id=WHITW3MM22QUDV24SYXS6AOBNZRI7YVYWVVKMS22BXQIJ3A74RHQ, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.282948549 (253h56m50.242045808s) [noise=true , address=104.36.82.60:4372 , piece_id=PR4H5TOHVFIP7DYLGHXI5FMSSSERZR4VBNVL577PZ4RPRKWGBPLA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.282990354 (253h56m50.242087601s) [noise=true , address=79.127.219.47:4062 , piece_id=7GPMR3NQNUHMIBRV3ITEFDXYB7CNJ3CBYJ4OCIJ6U4QWTNCQ4I6Q, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWXxGocse7Wur1vwWUY4e832MJq1L7BYBf4psNP6RpJTSJuMB): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.283071625 (253h56m50.242168871s) [noise=true , address=77.90.31.13:4631 , piece_id=WYKANSN542ACDRKAG34Q2QHIH2745W5IVRY63BFTPXRF4VCNJ3ZA, seg_pos={0 0}, upload=264353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aB5TGJszYj5bvk2cDbiQA7dcAoTxeYMsYgMWkkn1tACCxdtpz): protocol: expected piece hash; context canceled; EOF Aug 19 07:22:28.283094976 (253h56m50.242192221s) [seg_pos={0 0}, upload=264353]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 07:22:28.283189406 (253h56m50.242286641s) [upload=264353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 07:22:28.283198033 (253h56m50.242295278s) [upload=264353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:22:28.426563808 (253h56m50.385661094s) [upload=264353]: Done uploading segments... Aug 19 07:22:28.426606287 (253h56m50.385703533s) [upload=264353]: Upload finished. err: <nil> Duration: 2.327038699s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.094288491s FAIL (2025-08-19 07:21:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.094325703s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.708702975s FAIL (2025-08-19 07:21:56 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.708761272s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 834.986313ms FAIL (2025-08-19 07:21:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 835.01801ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.437464695s FAIL (2025-08-19 07:21:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.437494354s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 825.339047ms FAIL (2025-08-19 07:21:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 825.448499ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.436059206s FAIL (2025-08-19 07:20:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.436096513s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.040127104s FAIL (2025-08-19 07:20:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.040157727s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 935.188315ms FAIL (2025-08-19 07:20:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 935.275902ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.382916621s FAIL (2025-08-19 07:20:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.382951431s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.115287458s FAIL (2025-08-19 07:20:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.115321452s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 944.211165ms FAIL (2025-08-19 07:20:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 944.249125ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.387643965s FAIL (2025-08-19 07:19:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.387676763s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 815.863104ms FAIL (2025-08-19 07:19:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 815.895903ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.058456916s FAIL (2025-08-19 07:19:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.058558288s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 864.797948ms FAIL (2025-08-19 07:19:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 864.914941ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.006764538s FAIL (2025-08-19 07:18:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.006807371s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 870.258503ms FAIL (2025-08-19 07:18:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 870.352064ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.54026188s FAIL (2025-08-19 07:18:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.540288851s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.381859096s FAIL (2025-08-19 07:18:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.381958424s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X............................................X..............] duration: 808.92784ms FAIL (2025-08-19 07:18:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 809.027487ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 849.791097ms FAIL (2025-08-19 07:18:13 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 849.823749ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X............................................X.............] duration: 922.167497ms FAIL (2025-08-19 07:17:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 922.205695ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.249644806s FAIL (2025-08-19 07:17:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.249697362s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.445484768s FAIL (2025-08-19 07:17:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.44552757s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 876.983103ms FAIL (2025-08-19 07:17:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 877.071883ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X............................................X............] duration: 807.768264ms FAIL (2025-08-19 07:16:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 807.864423ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.503403764s FAIL (2025-08-19 07:16:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.503436734s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 871.25924ms FAIL (2025-08-19 07:16:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 871.291479ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X............................................X...........] duration: 956.557164ms FAIL (2025-08-19 07:16:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 956.585252ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.043774621s FAIL (2025-08-19 07:15:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.043828026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X............................................X..........] duration: 1.005116099s FAIL (2025-08-19 07:15:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.005153955s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.446291422s FAIL (2025-08-19 07:15:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.446327004s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.061137674s FAIL (2025-08-19 07:15:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 1.061173605s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.310395709s FAIL (2025-08-19 07:14:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.310486045s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X............................................X.........] duration: 814.018339ms FAIL (2025-08-19 07:14:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 814.046143ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 868.395614ms FAIL (2025-08-19 07:14:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 868.493289ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 953.356649ms FAIL (2025-08-19 07:13:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 953.392662ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X............................................X........] duration: 1.121506478s FAIL (2025-08-19 07:13:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 07:07:19.377746334 (253h41m41.33684357s) [upload=264078]: Starting upload Aug 19 07:07:19.377814342 (253h41m41.336911587s) [upload=264078]: Done starting upload Aug 19 07:07:19.377949333 (253h41m41.337046586s) [upload=264078]: Uploading segments... Aug 19 07:07:19.378717396 (253h41m41.337814675s) [upload=264078]: Got next segment. Inline: true Aug 19 07:07:19.378728290 (253h41m41.337825546s) [upload=264078]: Waiting for error group managing segments... Aug 19 07:07:19.378806794 (253h41m41.337904046s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 07:07:19.378813065 (253h41m41.337910318s) [upload=264078]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 07:07:19.378817315 (253h41m41.337914569s) [upload=264078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 07:07:19.466789521 (253h41m41.425886757s) [upload=264078]: Done uploading segments... Aug 19 07:07:19.466823399 (253h41m41.425920638s) [upload=264078]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.12153991s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.707997026s FAIL (2025-08-19 07:13:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 06:50:58.790241436 (253h25m20.749338681s) [upload=263801]: Starting upload Aug 19 06:50:58.790305933 (253h25m20.749403183s) [upload=263801]: Done starting upload Aug 19 06:50:58.790381720 (253h25m20.749478955s) [upload=263801]: Uploading segments... Aug 19 06:50:58.791004767 (253h25m20.750102022s) [upload=263801]: Got next segment. Inline: false Aug 19 06:50:58.791309050 (253h25m20.750406301s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:50:58.791319090 (253h25m20.750416345s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 06:50:58.989117311 (253h25m20.948214565s) [seg_pos={0 0}, upload=263801]: Begin upload segment... Aug 19 06:50:58.989206718 (253h25m20.948303976s) [seg_pos={0 0}, upload=263801]: Done begin upload segment. Aug 19 06:50:58.989259496 (253h25m20.948356745s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989310782 (253h25m20.948408065s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989713424 (253h25m20.94881066s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.989722404 (253h25m20.948819654s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990098221 (253h25m20.949195542s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990175739 (253h25m20.949272988s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990429951 (253h25m20.949527196s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990552196 (253h25m20.949649441s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990773433 (253h25m20.949870679s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.990905056 (253h25m20.950002304s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991100235 (253h25m20.950197483s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991341988 (253h25m20.950439236s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991459432 (253h25m20.950556679s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991759120 (253h25m20.950856369s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.991771915 (253h25m20.950869154s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992178676 (253h25m20.951275924s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.992652502 (253h25m20.951749747s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:58.995581757 (253h25m20.954679009s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Uploading piece... Aug 19 06:50:59.518391733 (253h25m21.477488972s) [upload=263801]: Next returned nil segment Aug 19 06:50:59.518411767 (253h25m21.477509014s) [upload=263801]: Waiting for error group managing segments... Aug 19 06:50:59.698014112 (253h25m21.657111362s) [noise=true , address=45.91.116.12:4235 , piece_id=KV4FUVLOHGV6EYOPXU7YZ5NBAJ7XJGUFEXP6VCCP7OMO6XFYWADQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.702749037 (253h25m21.661846282s) [noise=true , address=45.91.116.6:4990 , piece_id=7UTW5UIGZXJ5UVQSA3TC4ZACR5IFE7GM77M22ACMYJIGPQKJK65Q, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.703357129 (253h25m21.662454366s) [noise=true , address=77.90.31.18:4821 , piece_id=54HHSCTRMOGZXR77DY6KM53UXCCL2I23NFF6KJ3E25KJYNJR7MDA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.707342675 (253h25m21.666439961s) [noise=true , address=77.90.31.21:4908 , piece_id=6RR5LLC4V24LA6R464SUFJ5OOK5774OPYJ5HF6CRYQ3VI2CHJQHQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.709945235 (253h25m21.669042481s) [noise=true , address=45.91.116.11:4133 , piece_id=U3JEMTU7Y5YBOLP5B54VONM66XYXEUWWVBSMVRPDGQLWXZFOC72A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.711213294 (253h25m21.670310541s) [noise=true , address=77.90.31.17:4773 , piece_id=65NZOT6IREIMMDLNPOVJ4HEO76VCBCDDPY7PVHGGW6XOMRUPUNCA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.730511290 (253h25m21.689608537s) [noise=true , address=104.36.82.34:4569 , piece_id=6LS6I7TKAJBTZD6SYUKNDUFHJSUZJZ7KWOQMROMKFNEZBGYASWGQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.734755082 (253h25m21.693852329s) [noise=true , address=45.91.116.9:4286 , piece_id=4G2CZUEF6AGMKGNA2X7SQ7YUJBYIXSIEKERLZY4RTITHQFAPYTOQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.737779353 (253h25m21.696876593s) [noise=true , address=77.90.31.10:4529 , piece_id=QR73ODSPOWO2CTQF6SFKQNB5P4A6TRVLEUQICNGVJY33RAZNZVOA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.756559828 (253h25m21.71565707s) [noise=true , address=77.90.31.11:4568 , piece_id=TLTFR4J6WNXO7BCJA3IHDYSDJXO6ZW24JBCFJY4LKCJB4HM2XNXQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.764835289 (253h25m21.723932542s) [noise=true , address=79.127.219.51:4205 , piece_id=V6S6QZNNARQ5RCS7DHOPJHJ4TOSLUBCDXUJAHY7ENXFNKVVH4YTA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.778410505 (253h25m21.737507755s) [noise=true , address=45.91.116.13:4157 , piece_id=SAON2W5KS5L5335HR2KD7LGD2VO3XJ7YDB4JXG67HITZOKXTQ6QQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.832485490 (253h25m21.791582737s) [noise=true , address=104.36.82.49:4011 , piece_id=DIPKNERR2VBYPGK4BLF6DCA7YBO3WKSQGQADKZTABMXKYXZ47MJQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871006105 (253h25m21.830103355s) [noise=true , address=104.36.82.23:4459 , piece_id=G5X7WHXX7KOHVYVCRVCO37EGQ674PCNVY2UVAN36U7TBQHFJF6CQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: <nil> Aug 19 06:50:59.871044967 (253h25m21.830142212s) [seg_pos={0 0}, upload=263801]: Segment reached optimal threshold of 14 pieces. Aug 19 06:50:59.871244725 (253h25m21.830341971s) [noise=true , address=79.127.201.215:4360 , piece_id=B4RTBLD5PH74C5446FBLU26PQFV3PL3XJ6PGSS63U52MXUV3IL5A, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871495889 (253h25m21.830593143s) [noise=true , address=104.36.82.60:4361 , piece_id=3LRDGGHCOFS3ABEF4PAGINCKEQJQ3BQP65MQCZ52UMCWF4SF5ETA, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbxCfwkBdYcJG3LKJnU19Kyk9DtkhKdEAKoVfL353zC6Hge2k): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871619034 (253h25m21.830716265s) [noise=true , address=79.127.220.101:4041 , piece_id=HBXSP5AU44AZRN6CHDQC5WI44DQVMY2GSZLI3IZIMTUMYY2SNWCQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw): protocol: expected piece hash; context canceled; EOF Aug 19 06:50:59.871785467 (253h25m21.8308827s) [noise=true , address=66.23.193.175:4553 , piece_id=L5F7GUL5SOXDZ3RJ5FB64CXWGJHR5E6YSN4ZJNYPZJNW26NB7SUQ, seg_pos={0 0}, upload=263801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14ZJ9cf9uqJqtXTkP9VL2KU5YdCJfUVSyJkfRwJ6K7SpRB5zME): protocol: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34518->66.23.193.175:4553: use of closed network connection Aug 19 06:50:59.871804083 (253h25m21.83090133s) [seg_pos={0 0}, upload=263801]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 06:50:59.871904895 (253h25m21.831002145s) [upload=263801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 06:50:59.871910790 (253h25m21.831008043s) [upload=263801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:51:00.028627355 (253h25m21.987724607s) [upload=263801]: Done uploading segments... Aug 19 06:51:00.028666508 (253h25m21.987763759s) [upload=263801]: Upload finished. err: <nil> Duration: 2.708031643s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 845.918568ms FAIL (2025-08-19 07:13:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 845.947224ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 853.753231ms FAIL (2025-08-19 07:12:54 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 06:38:44.764639559 (253h13m6.723736805s) [upload=263587]: Starting upload Aug 19 06:38:44.764722805 (253h13m6.723820074s) [upload=263587]: Done starting upload Aug 19 06:38:44.764968851 (253h13m6.724066098s) [upload=263587]: Uploading segments... Aug 19 06:38:44.765051465 (253h13m6.72414872s) [upload=263587]: Got next segment. Inline: true Aug 19 06:38:44.765058669 (253h13m6.724155915s) [upload=263587]: Waiting for error group managing segments... Aug 19 06:38:44.765130283 (253h13m6.724227532s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 06:38:44.765136103 (253h13m6.72423336s) [upload=263587]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 06:38:44.765140387 (253h13m6.724237637s) [upload=263587]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 06:38:44.919946054 (253h13m6.879043293s) [upload=263587]: Done uploading segments... Aug 19 06:38:44.919980537 (253h13m6.879077782s) [upload=263587]: Upload finished. err: <nil> Duration:* 853.791823ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)