Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................................XXX.XXX............] duration: 2.374666519s FAIL (2025-07-11 22:16:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.374705131s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..........................................XX..XX............] duration: 3.072211861s FAIL (2025-07-11 22:15:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.072243266s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................................XXX.XXX...........] duration: 2.358879378s FAIL (2025-07-11 22:15:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.358909694s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [........................................XXX.XXXXX...........] duration: 2.74662542s FAIL (2025-07-11 22:15:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.746658091s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................................XXX.XXX..........] duration: 2.392068693s FAIL (2025-07-11 22:15:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.392100432s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...........................................XX..XX...........] duration: 2.597748724s FAIL (2025-07-11 22:15:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.597783362s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.........................................XXX.XXXXX..........] duration: 2.60309261s FAIL (2025-07-11 22:15:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.603159593s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [............................................XX..XX..........] duration: 2.55928788s FAIL (2025-07-11 22:14:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.55931927s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........................................XXX.XXXXX.........] duration: 2.690135644s FAIL (2025-07-11 22:14:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.690161322s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................................XXX.XXX.........] duration: 2.380011101s FAIL (2025-07-11 22:14:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.380043856s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...........................................XXX.XXXXX........] duration: 2.764153824s FAIL (2025-07-11 22:14:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.76428035s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.............................................XX..XX.........] duration: 3.748301457s FAIL (2025-07-11 22:13:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.748335064s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................................XXX.XXX........] duration: 2.438633135s FAIL (2025-07-11 22:13:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.438661658s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [............................................XXX.XXXXX.......] duration: 2.812094563s FAIL (2025-07-11 22:13:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.812131404s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..............................................XX..XX........] duration: 2.540210992s FAIL (2025-07-11 22:12:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.540246066s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.............................................XXX.XXXXX......] duration: 2.913955601s FAIL (2025-07-11 22:12:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.913978792s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................................XXX.XXX.......] duration: 2.349989249s FAIL (2025-07-11 22:12:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.350021319s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XX..XX.......] duration: 3.972097493s FAIL (2025-07-11 22:12:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.972126427s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..............................................XXX.XXXXX.....] duration: 2.780756657s FAIL (2025-07-11 22:12:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.780805347s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................................XXX.XXX......] duration: 2.392752461s FAIL (2025-07-11 22:11:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.392786942s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...............................................XXX.XXXXX....] duration: 2.62640629s FAIL (2025-07-11 22:11:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.626431536s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................................XXX.XXX.....] duration: 2.525109614s FAIL (2025-07-11 22:11:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.525145809s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [................................................XX..XX......] duration: 3.25868188s FAIL (2025-07-11 22:11:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.258718469s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [................................................XXX.XXXXX...] duration: 2.811604995s FAIL (2025-07-11 22:11:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.811636902s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................................XXX.XXX....] duration: 2.766503963s FAIL (2025-07-11 22:10:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.766610641s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XX..XX.....] duration: 3.927224619s FAIL (2025-07-11 22:10:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.927257835s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.................................................XXX.XXXXX..] duration: 3.76694299s FAIL (2025-07-11 22:10:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.767014069s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................................................XXX.XXX...] duration: 5.038438156s FAIL (2025-07-11 22:09:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.038475088s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..................................................XXX.XXXXX.] duration: 10.460735713s FAIL (2025-07-11 22:09:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.460777727s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XX..XX....] duration: 11.036494408s FAIL (2025-07-11 22:09:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 11.036533724s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................................................XXX.XXX..] duration: 3.043309734s FAIL (2025-07-11 22:09:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 3.043345658s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...................................................XXX.XXXXX] duration: 30.000675304s FAIL (2025-07-11 22:08:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000698795s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...................................................XX..XX...] duration: 3.291728941s FAIL (2025-07-11 22:08:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.291760848s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................................XXX.XXX.] duration: 2.930020243s FAIL (2025-07-11 22:08:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.930054273s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XXX.XXX] duration: 30.114065208s FAIL (2025-07-11 22:07:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.114111233s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....................................................XXX.XXXX] duration: 30.001250113s FAIL (2025-07-11 22:07:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001271465s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XX..XX..] duration: 7.005789711s FAIL (2025-07-11 22:07:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.005816352s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................XX..XX.] duration: 15.799146717s FAIL (2025-07-11 22:07:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 15.799169372s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XXX.XXX] duration: 30.001494014s FAIL (2025-07-11 22:07:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001510532s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XX..XX] duration: 1m0.119137317s FAIL (2025-07-11 22:06:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.119155832s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XXX.XX] duration: 30.105881644s FAIL (2025-07-11 22:06:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== UE7N7RBTDETKFO2VBQHMTQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.133666060 (444h55m5.017079696s) [noise=true , address=104.36.82.9:4005 , piece_id=PPFUSA2W6GEWONBYBRT52YZEURQATGSDVERB7NW5FI4TEGLO33ZA, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.134087567 (444h55m5.017501209s) [noise=true , address=104.36.82.13:4187 , piece_id=64IFFQLSGZL6K2Z5V4J4COODY7KXKHVOQOEGOZZBDJW3RGWJDSEA, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.134256035 (444h55m5.017669668s) [noise=true , address=77.90.31.17:4772 , piece_id=4UHYRDGZHM7FPD7ZK46SFXI54HRMNBWISG53OJWVLPPIJHG7PUDQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.134495636 (444h55m5.017909288s) [noise=true , address=104.36.82.3:4767 , piece_id=KUQBH2WEMXX6Y42ILGHDNIEKE6OVTX2F3XK5X4IHXTRODLILJDSQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.134897654 (444h55m5.018311287s) [noise=true , address=104.36.82.10:4078 , piece_id=5MVTGTPKB3IOV4YVQB4KGMC7JKQRGYIBKCHP4LKGLORG34HYJDIQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.135292900 (444h55m5.018706535s) [noise=true , address=104.36.82.15:4258 , piece_id=LAS3CRJZDRQZLMOF22ETEVZ7O67NJJE4YE2UB7RFY5IDDRGWYQ6A, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.135552707 (444h55m5.018966337s) [noise=true , address=104.36.82.12:4151 , piece_id=4YDLARZNKEKGHKZXNDNM3JBNA3CPADNSTD7UAMDUU4BANDGIUDFQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.135900233 (444h55m5.01931386s) [noise=true , address=77.90.31.13:4626 , piece_id=OUT476GEFNOWZWOJXRFFBZY6TJSUYSE6ZBZFJXQAC4OKVGBGUWOQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.136252343 (444h55m5.019665975s) [noise=true , address=104.36.82.11:4115 , piece_id=GH6QVTPGIF554M7RTL3CJGTVB4SS444DVNGJWVCGJ6XJKVQ6N2OQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.136498346 (444h55m5.019911992s) [noise=true , address=77.90.31.11:4554 , piece_id=RF3JPVM74Q5BT4XNKZLPASES7ZRMN75THDXVIHZTUWZPHQI7R2BA, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.136900343 (444h55m5.020313979s) [noise=true , address=104.36.82.7:4911 , piece_id=ONQAL7WGOQZFGCWGR3DTWRSTWAX2TPM2YFLHMSMO3RHU3HSVYA3A, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.137291378 (444h55m5.020705006s) [noise=true , address=45.91.116.4:4939 , piece_id=FN2DHKEHLNHDJRYJPY73D447FCQLFCNSSAGES3DYXRI4ISXJSYZA, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.137399682 (444h55m5.020813312s) [noise=true , address=104.36.82.6:4876 , piece_id=RLEFABSMYPGIUS4JHOTJL6GZGEOPPJLV3ZRA5CZKCBPYEO5ZQRAQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.137712035 (444h55m5.021125668s) [noise=true , address=104.36.82.16:4295 , piece_id=CNHBOFILZ2EFPQI5R4VJJYK2L4YNX4B6RKXI73OOZUYKRZBM2L6A, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.138076815 (444h55m5.021490442s) [noise=true , address=77.90.31.10:4518 , piece_id=MJDPMI2GFHTNIZ24CDPDPBCK6GZ3LEKRZCU4FAPMDA7ZUK5EW3AQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:06.709355010 (444h55m6.59276863s) [upload=355787]: Next returned nil segment Jul 11 22:06:06.709388206 (444h55m6.592801833s) [upload=355787]: Waiting for error group managing segments... Jul 11 22:06:08.096853372 (444h55m7.980267047s) [noise=true , address=77.90.31.11:4554 , piece_id=RF3JPVM74Q5BT4XNKZLPASES7ZRMN75THDXVIHZTUWZPHQI7R2BA, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:10.196362260 (444h55m10.079775872s) [noise=true , address=77.90.31.13:4626 , piece_id=OUT476GEFNOWZWOJXRFFBZY6TJSUYSE6ZBZFJXQAC4OKVGBGUWOQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:15.218285149 (444h55m15.101698768s) [noise=true , address=77.90.31.10:4518 , piece_id=MJDPMI2GFHTNIZ24CDPDPBCK6GZ3LEKRZCU4FAPMDA7ZUK5EW3AQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:16.129501415 (444h55m16.012915049s) [noise=true , address=77.90.31.17:4772 , piece_id=4UHYRDGZHM7FPD7ZK46SFXI54HRMNBWISG53OJWVLPPIJHG7PUDQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:17.385731005 (444h55m17.26914464s) [noise=true , address=45.91.116.4:4939 , piece_id=FN2DHKEHLNHDJRYJPY73D447FCQLFCNSSAGES3DYXRI4ISXJSYZA, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:34.749342136 (444h55m34.632755751s) [noise=true , address=104.36.82.3:4767 , piece_id=KUQBH2WEMXX6Y42ILGHDNIEKE6OVTX2F3XK5X4IHXTRODLILJDSQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749436984 (444h55m34.632850611s) [noise=true , address=104.36.82.12:4151 , piece_id=4YDLARZNKEKGHKZXNDNM3JBNA3CPADNSTD7UAMDUU4BANDGIUDFQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749543804 (444h55m34.632957422s) [noise=true , address=104.36.82.13:4187 , piece_id=64IFFQLSGZL6K2Z5V4J4COODY7KXKHVOQOEGOZZBDJW3RGWJDSEA, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: write tcp 139.59.144.22:37398->104.36.82.13:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749610349 (444h55m34.633023965s) [noise=true , address=104.36.82.7:4911 , piece_id=ONQAL7WGOQZFGCWGR3DTWRSTWAX2TPM2YFLHMSMO3RHU3HSVYA3A, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749661721 (444h55m34.633075338s) [noise=true , address=104.36.82.6:4876 , piece_id=RLEFABSMYPGIUS4JHOTJL6GZGEOPPJLV3ZRA5CZKCBPYEO5ZQRAQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:41804->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749712162 (444h55m34.633125778s) [noise=true , address=104.36.82.14:4221 , piece_id=K4I4QC4YKAW6M5UXVLNKTPCKEI6NQNUE7N7RBTDETKFO2VBQHMTQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: write tcp 139.59.144.22:41990->104.36.82.14:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749863225 (444h55m34.633276845s) [noise=true , address=104.36.82.9:4005 , piece_id=PPFUSA2W6GEWONBYBRT52YZEURQATGSDVERB7NW5FI4TEGLO33ZA, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 139.59.144.22:48210->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749996307 (444h55m34.633409933s) [noise=true , address=104.36.82.10:4078 , piece_id=5MVTGTPKB3IOV4YVQB4KGMC7JKQRGYIBKCHP4LKGLORG34HYJDIQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: rpc: dial tcp 104.36.82.10:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.750038894 (444h55m34.63345251s) [noise=true , address=104.36.82.16:4295 , piece_id=CNHBOFILZ2EFPQI5R4VJJYK2L4YNX4B6RKXI73OOZUYKRZBM2L6A, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 139.59.144.22:49444->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.750048359 (444h55m34.633461975s) [noise=true , address=104.36.82.11:4115 , piece_id=GH6QVTPGIF554M7RTL3CJGTVB4SS444DVNGJWVCGJ6XJKVQ6N2OQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 139.59.144.22:38672->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.750091039 (444h55m34.633504657s) [noise=true , address=104.36.82.15:4258 , piece_id=LAS3CRJZDRQZLMOF22ETEVZ7O67NJJE4YE2UB7RFY5IDDRGWYQ6A, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:47840->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.750126727 (444h55m34.633540353s) [seg_pos={0 0}, upload=355787]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 11 22:06:34.853461811 (444h55m34.736875429s) [upload=355787]: Done uploading segments... Jul 11 22:06:34.853512446 (444h55m34.736926063s) [upload=355787]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.105909107s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XXX.XX] duration: 30.000833441s FAIL (2025-07-11 22:06:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000860292s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XX..X] duration: 1m0.106845319s FAIL (2025-07-11 22:05:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== .. Jul 11 22:04:55.072689878 (444h53m54.956103505s) [upload=355771]: Got next segment. Inline: false Jul 11 22:04:55.072753570 (444h53m54.956167196s) [upload=355771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:04:55.072759932 (444h53m54.956173558s) [upload=355771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:04:55.313585304 (444h53m55.19699899s) [seg_pos={0 0}, upload=355771]: Begin upload segment... Jul 11 22:04:55.313726244 (444h53m55.197139859s) [noise=true , address=77.90.31.15:4730 , piece_id=APUA6M3ZLFKYXNUUU2SOQ4MUA7YVDEGWN6H6FXV5NWCJSJFMPSBA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.313758278 (444h53m55.197171913s) [noise=true , address=104.36.82.39:4735 , piece_id=6TN4HDENV6FZXRU6MXF623TVZUSBB6NJAA2JWDNFLV3CF565XG2A, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.314133360 (444h53m55.197546995s) [noise=true , address=45.91.116.5:4304 , piece_id=LVPWMZCOTZQRCUYFWVTRA5BWKVRDHS2ZUNZTJKJGWACUWOKM3SCQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.314369395 (444h53m55.197783007s) [seg_pos={0 0}, upload=355771]: Done begin upload segment. Jul 11 22:04:55.314377578 (444h53m55.197791191s) [noise=true , address=45.91.95.2:4018 , piece_id=QH5M55Q2R6OC4IHELLZQRZXTPQK5MIOBJOXQ3MATWRIUEZPP4HMA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.314387799 (444h53m55.197801426s) [noise=true , address=45.91.95.3:4004 , piece_id=5PMG6JYVOSW2FT2Y47QL2E5YVI3A54TV65SMHMQMG3OQX5XD5U5Q, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.314696651 (444h53m55.198110279s) [noise=true , address=77.90.31.17:4769 , piece_id=XVHHMBMH3SYRXDLTSRUSE3OUKHYEH5QNU4Y6PQGDQXUT5FXK4TNA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.315247269 (444h53m55.198660893s) [noise=true , address=77.90.31.13:4629 , piece_id=LDXOLEO34KPI5LG5FSNAOIWV2K5AJOXGNRILNJJOO4KQZ6QBU3SQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.318188723 (444h53m55.201602358s) [noise=true , address=79.127.219.38:4035 , piece_id=IOLSZGLRTYUA37HWFGCFO4CXQATUHBLTCE2HPVNDDJRAQ3L3KFPA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.318647317 (444h53m55.202060946s) [noise=true , address=77.90.31.19:4853 , piece_id=7O7DDEURX4VA5JBNETWGAJWGT244MFNTPMNJG33KJFOHWYWUSVAA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.319019338 (444h53m55.202432967s) [noise=true , address=79.127.205.234:4598 , piece_id=HDCUJRXFWHSGNYBXM67BQESYXHTHBLRT2E2SH43U6SKBBXZ54EKA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.319406715 (444h53m55.202820351s) [noise=true , address=104.36.82.14:4227 , piece_id=XGVFZPSGVK2OVWY7HJC2AHISRZBSYEM6FFZGHBP5OLX5HU44NGTA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.320615595 (444h53m55.204029249s) [noise=true , address=104.36.82.51:4053 , piece_id=4UYZRIMBWYUOA42PJ2TLAO2IOGIIXEWCBDNQWCPZOHKKOFJKCAQA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.321201069 (444h53m55.204614709s) [noise=true , address=104.36.82.18:4396 , piece_id=TQ37GIZZ4DBLH2XVNWQV4QXY4K3AJ5NXFP6RZJ32OZKAASPNEVLA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.321816892 (444h53m55.205230528s) [noise=true , address=45.91.95.4:4008 , piece_id=QCG67FGPEPSYXK22CLKPQRQXZCPEB5PI2M4MOQYZLKYR4C33WPGQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.322188584 (444h53m55.205602214s) [noise=true , address=104.36.82.40:4965 , piece_id=2KFI3M4DBLMBXNALUZG3X2GBXLD3RHDOYPEX4E72WGP5D43YR46Q, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.322553656 (444h53m55.205967285s) [noise=true , address=45.91.116.12:4252 , piece_id=JGSTDTCUZ53KDL4MEHNZ5NMMNHV6IG7JVFQNRA76O5G6BMFMNUCQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.323220751 (444h53m55.206634384s) [noise=true , address=77.90.31.18:4811 , piece_id=7CJCDEFU4523D4YHMSFQPA7STYFYUG4VZQRPMICNCSBULG6DLRWQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.323528348 (444h53m55.206941974s) [noise=true , address=77.90.31.12:4605 , piece_id=K7AUJNYTUNSRA6UBDPQI2DF3QEI4X3HKG6ANOK5NAUXTGJRHJEDQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.902170512 (444h53m55.785584152s) [upload=355771]: Next returned nil segment Jul 11 22:04:55.902218409 (444h53m55.785632036s) [upload=355771]: Waiting for error group managing segments... Jul 11 22:04:56.598198976 (444h53m56.4816126s) [noise=true , address=79.127.205.234:4598 , piece_id=HDCUJRXFWHSGNYBXM67BQESYXHTHBLRT2E2SH43U6SKBBXZ54EKA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:04:57.441463597 (444h53m57.324877217s) [noise=true , address=77.90.31.15:4730 , piece_id=APUA6M3ZLFKYXNUUU2SOQ4MUA7YVDEGWN6H6FXV5NWCJSJFMPSBA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:04:57.617801779 (444h53m57.501215406s) [noise=true , address=45.91.116.5:4304 , piece_id=LVPWMZCOTZQRCUYFWVTRA5BWKVRDHS2ZUNZTJKJGWACUWOKM3SCQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:04:59.206776760 (444h53m59.090190379s) [noise=true , address=77.90.31.17:4769 , piece_id=XVHHMBMH3SYRXDLTSRUSE3OUKHYEH5QNU4Y6PQGDQXUT5FXK4TNA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:04:59.806024373 (444h53m59.689438009s) [noise=true , address=77.90.31.18:4811 , piece_id=7CJCDEFU4523D4YHMSFQPA7STYFYUG4VZQRPMICNCSBULG6DLRWQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:00.604640851 (444h54m0.488054478s) [noise=true , address=77.90.31.19:4853 , piece_id=7O7DDEURX4VA5JBNETWGAJWGT244MFNTPMNJG33KJFOHWYWUSVAA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:02.621235083 (444h54m2.5046487s) [noise=true , address=79.127.219.38:4035 , piece_id=IOLSZGLRTYUA37HWFGCFO4CXQATUHBLTCE2HPVNDDJRAQ3L3KFPA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:04.045177388 (444h54m3.92859102s) [noise=true , address=77.90.31.12:4605 , piece_id=K7AUJNYTUNSRA6UBDPQI2DF3QEI4X3HKG6ANOK5NAUXTGJRHJEDQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:04.409180174 (444h54m4.292593788s) [noise=true , address=45.91.116.12:4252 , piece_id=JGSTDTCUZ53KDL4MEHNZ5NMMNHV6IG7JVFQNRA76O5G6BMFMNUCQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:07.797191258 (444h54m7.680604874s) [noise=true , address=77.90.31.13:4629 , piece_id=LDXOLEO34KPI5LG5FSNAOIWV2K5AJOXGNRILNJJOO4KQZ6QBU3SQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:11.085763116 (444h54m10.969176745s) [noise=true , address=45.91.95.3:4004 , piece_id=5PMG6JYVOSW2FT2Y47QL2E5YVI3A54TV65SMHMQMG3OQX5XD5U5Q, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:28.810582170 (444h54m28.693995785s) [noise=true , address=45.91.95.2:4018 , piece_id=QH5M55Q2R6OC4IHELLZQRZXTPQK5MIOBJOXQ3MATWRIUEZPP4HMA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:51.180763268 (444h54m51.064176887s) [noise=true , address=45.91.95.4:4008 , piece_id=QCG67FGPEPSYXK22CLKPQRQXZCPEB5PI2M4MOQYZLKYR4C33WPGQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:54.802082913 (444h54m54.685496543s) [noise=true , address=104.36.82.39:4735 , piece_id=6TN4HDENV6FZXRU6MXF623TVZUSBB6NJAA2JWDNFLV3CF565XG2A, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:1QCQZ8eFGLa5TAaAJJzUip8o5DAELzTBTm4UFbt3Am5yYWXAZF, address:104.36.82.39:4735): protocol: rpc: dial tcp 104.36.82.39:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:54.802160234 (444h54m54.685573849s) [noise=true , address=104.36.82.51:4053 , piece_id=4UYZRIMBWYUOA42PJ2TLAO2IOGIIXEWCBDNQWCPZOHKKOFJKCAQA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG, address:104.36.82.51:4053): protocol: rpc: dial tcp 104.36.82.51:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:54.802233684 (444h54m54.685647315s) [noise=true , address=104.36.82.40:4965 , piece_id=2KFI3M4DBLMBXNALUZG3X2GBXLD3RHDOYPEX4E72WGP5D43YR46Q, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:1ToFTqjRAWvpRQzf6vyHKDpforPyb4LbDGcCVhRhkwiBGKihnD, address:104.36.82.40:4965): protocol: rpc: dial tcp 104.36.82.40:4965: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:54.802337037 (444h54m54.685750651s) [noise=true , address=104.36.82.14:4227 , piece_id=XGVFZPSGVK2OVWY7HJC2AHISRZBSYEM6FFZGHBP5OLX5HU44NGTA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:104.36.82.14:4227): protocol: rpc: dial tcp 104.36.82.14:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:54.802419137 (444h54m54.685832764s) [noise=true , address=104.36.82.18:4396 , piece_id=TQ37GIZZ4DBLH2XVNWQV4QXY4K3AJ5NXFP6RZJ32OZKAASPNEVLA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx, address:104.36.82.18:4396): protocol: write tcp 139.59.144.22:57994->104.36.82.18:4396: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Jul 11 22:05:54.802441878 (444h54m54.685855512s) [seg_pos={0 0}, upload=355771]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:05:54.906760648 (444h54m54.790174283s) [upload=355771]: Done uploading segments... Jul 11 22:05:54.906793413 (444h54m54.790207044s) [upload=355771]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.106867521s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXX.X] duration: 30.109352013s FAIL (2025-07-11 22:05:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== B3TRY7DESLM7RITRW3N25NU6RSLNMS53YTKG5M3ICYHL6A, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.648097583 (444h53m59.531511213s) [noise=true , address=104.36.82.16:4295 , piece_id=HAEASJ6R5JE3C7KYMP4VQ5TZEKOF33XADVXFEBAC3XNVI2EZY2YA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.648622050 (444h53m59.532035684s) [noise=true , address=77.90.31.19:4843 , piece_id=6TXPOBW44VUWY2HU33TZ4VILNJJORJ5UCNCRCDVVOH3RCC3RXVGA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649033035 (444h53m59.532446676s) [noise=true , address=104.36.82.15:4258 , piece_id=JXFOVVJSVLUOI3XMRAWICOKUVSLGPSNZPT7S3RZFSCZQP74GM7VA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649060162 (444h53m59.532473787s) [noise=true , address=45.91.116.13:4153 , piece_id=CUHA4TL64SWHYZ6XU5C7RGUPABM4N26YVEOO6NA7PS6ZN4ZLWFWQ, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649485981 (444h53m59.532899616s) [noise=true , address=77.90.31.18:4807 , piece_id=O5VMFRIB776MOJSNCGQ56NJDBNTA37CUECNLU766TAIRK766XS4Q, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649688620 (444h53m59.533102257s) [noise=true , address=104.36.82.10:4078 , piece_id=6FXQSLTPKPLXHKUKRWTP2SVOCDGLG6VHQRQ7KASKOQOJSS4746XA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649855897 (444h53m59.533269532s) [noise=true , address=104.36.82.2:4732 , piece_id=UXLFRQF2NNCI4DGQYMVN5L3W7OXLUGA2MZOUTY3C7KKZFKUM4B4Q, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.650270507 (444h53m59.533684143s) [noise=true , address=104.36.82.9:4007 , piece_id=IA6UICMJEFAPRL7IUG5FEOB2YODPXR6UZEYCKZIXDAVBNVCB4PZA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.650278311 (444h53m59.533691947s) [noise=true , address=104.36.82.6:4875 , piece_id=U3T7QIIHRHPVY2RN3GMUL66JCWU5AYFFU4OWJBOGZB2SFVBHR7GA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.650316741 (444h53m59.533730353s) [noise=true , address=104.36.82.5:4840 , piece_id=FCU7E7ZA3KGTHYB3ZRW6SQAKYMMLLJYE7FROIJCRXIBW6EADUC3A, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.650974066 (444h53m59.534387741s) [noise=true , address=77.90.31.14:4663 , piece_id=AIFMRAEYZ2UBGGT5BBNP5RS2SQWUGTVWPE76J2H3HT4KY2ELBIKA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.651716428 (444h53m59.535130055s) [noise=true , address=104.36.82.8:4969 , piece_id=ZB7AL22XVG32DFXJFUETTJ7TICHM5FTRZLFWU2AYOLCBZNISUWPA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.652630735 (444h53m59.536044373s) [noise=true , address=45.91.116.7:4010 , piece_id=3WSD6MQW6G6CC2VKYGKFT7YEVE47X6OQAZBXTORDHEBTJ42SHUJQ, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.653573814 (444h53m59.536987442s) [noise=true , address=104.36.82.14:4223 , piece_id=RJ6HCZ5HCCFUQMBUP7KM5GMX24X2Y4VM3ADELWR6I5GGPAUO24QQ, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.654426947 (444h53m59.537840589s) [noise=true , address=104.36.82.16:4293 , piece_id=F4BALD5B6J4ITDPZ62AURSSNNGKWATJVPFC7BC7WDNS5SB2KIQBA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:05:06.634724509 (444h54m6.518138135s) [upload=355774]: Next returned nil segment Jul 11 22:05:06.634746228 (444h54m6.518159866s) [upload=355774]: Waiting for error group managing segments... Jul 11 22:05:12.832681435 (444h54m12.716095056s) [noise=true , address=77.90.31.14:4663 , piece_id=AIFMRAEYZ2UBGGT5BBNP5RS2SQWUGTVWPE76J2H3HT4KY2ELBIKA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:13.441666250 (444h54m13.325079877s) [noise=true , address=77.90.31.18:4807 , piece_id=O5VMFRIB776MOJSNCGQ56NJDBNTA37CUECNLU766TAIRK766XS4Q, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:13.629841060 (444h54m13.51325469s) [noise=true , address=45.91.116.13:4153 , piece_id=CUHA4TL64SWHYZ6XU5C7RGUPABM4N26YVEOO6NA7PS6ZN4ZLWFWQ, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:20.797771223 (444h54m20.681184839s) [noise=true , address=45.91.116.7:4010 , piece_id=3WSD6MQW6G6CC2VKYGKFT7YEVE47X6OQAZBXTORDHEBTJ42SHUJQ, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:22.776372688 (444h54m22.659786304s) [noise=true , address=77.90.31.19:4843 , piece_id=6TXPOBW44VUWY2HU33TZ4VILNJJORJ5UCNCRCDVVOH3RCC3RXVGA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:29.256798257 (444h54m29.140211877s) [noise=true , address=104.36.82.10:4078 , piece_id=6FXQSLTPKPLXHKUKRWTP2SVOCDGLG6VHQRQ7KASKOQOJSS4746XA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: rpc: dial tcp 104.36.82.10:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.256931538 (444h54m29.140345156s) [noise=true , address=104.36.82.2:4732 , piece_id=UXLFRQF2NNCI4DGQYMVN5L3W7OXLUGA2MZOUTY3C7KKZFKUM4B4Q, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: write tcp 139.59.144.22:58868->104.36.82.2:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257021976 (444h54m29.140435603s) [noise=true , address=104.36.82.9:4007 , piece_id=IA6UICMJEFAPRL7IUG5FEOB2YODPXR6UZEYCKZIXDAVBNVCB4PZA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257128785 (444h54m29.140542399s) [noise=true , address=104.36.82.3:4767 , piece_id=53FPKVB3TRY7DESLM7RITRW3N25NU6RSLNMS53YTKG5M3ICYHL6A, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:60582->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257150641 (444h54m29.140564253s) [noise=true , address=104.36.82.15:4258 , piece_id=JXFOVVJSVLUOI3XMRAWICOKUVSLGPSNZPT7S3RZFSCZQP74GM7VA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:50642->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257211975 (444h54m29.140625607s) [noise=true , address=104.36.82.16:4295 , piece_id=HAEASJ6R5JE3C7KYMP4VQ5TZEKOF33XADVXFEBAC3XNVI2EZY2YA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: rpc: dial tcp 104.36.82.16:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257305598 (444h54m29.140719227s) [noise=true , address=104.36.82.6:4875 , piece_id=U3T7QIIHRHPVY2RN3GMUL66JCWU5AYFFU4OWJBOGZB2SFVBHR7GA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: rpc: dial tcp 104.36.82.6:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257365719 (444h54m29.140779347s) [noise=true , address=104.36.82.8:4969 , piece_id=ZB7AL22XVG32DFXJFUETTJ7TICHM5FTRZLFWU2AYOLCBZNISUWPA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: rpc: dial tcp 104.36.82.8:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257479175 (444h54m29.140892803s) [noise=true , address=104.36.82.14:4223 , piece_id=RJ6HCZ5HCCFUQMBUP7KM5GMX24X2Y4VM3ADELWR6I5GGPAUO24QQ, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 139.59.144.22:52210->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257571361 (444h54m29.140984974s) [noise=true , address=104.36.82.16:4293 , piece_id=F4BALD5B6J4ITDPZ62AURSSNNGKWATJVPFC7BC7WDNS5SB2KIQBA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 139.59.144.22:51066->104.36.82.16:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257632322 (444h54m29.141045949s) [noise=true , address=104.36.82.5:4840 , piece_id=FCU7E7ZA3KGTHYB3ZRW6SQAKYMMLLJYE7FROIJCRXIBW6EADUC3A, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: write tcp 139.59.144.22:46436->104.36.82.5:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257655258 (444h54m29.141068894s) [seg_pos={0 0}, upload=355774]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 11 22:05:29.363237589 (444h54m29.246651207s) [upload=355774]: Done uploading segments... Jul 11 22:05:29.363263156 (444h54m29.246676782s) [upload=355774]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.109448291s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 18.089470142s FAIL (2025-07-11 22:05:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.08952218s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X......................................................XXX.X] duration: 30.001510929s FAIL (2025-07-11 22:05:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001900675s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 12.63924171s FAIL (2025-07-11 22:05:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.639268864s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 9.236778677s FAIL (2025-07-11 22:04:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 737s) [noise=true , address=45.91.95.2:4016 , piece_id=OB4WV3YWOQNVUQIUAKFKDS7NF7FX54UEPBZVNNEUAORZUFE2FTXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.488709574 (444h51m30.372123209s) [noise=true , address=104.36.82.50:4031 , piece_id=ZTD6S46WOAD2BKVVJ3RCHX55RGJNITAVPZ3ENGQBAQB3OPLW6C2A, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489246417 (444h51m30.372660054s) [noise=true , address=45.91.116.4:4966 , piece_id=K7JXTKTY4UJSCMJUOSWFWHIIZBNCXSP3N5K3INQDF77HDAKHRGIQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489303194 (444h51m30.372716825s) [noise=true , address=77.90.31.12:4617 , piece_id=WZWXQJGB4MTW4JN44NY5NQCXOWZQSGBX6IIBF56TQKB3UFQ7JEJA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489648268 (444h51m30.373061904s) [noise=true , address=77.90.31.16:4755 , piece_id=KI32C4HB2RCC4EDMM2B6UW6OUPSMAKU5GOA5ANIXTJQD3WLQLXBQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489707285 (444h51m30.373120911s) [noise=true , address=45.91.95.5:4002 , piece_id=3IZ3MAVFGNZO2C7LPTOY2FTIC6R2DC4OJY6XBJVWMPRE63B4DUEQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490095044 (444h51m30.373508684s) [noise=true , address=79.127.220.101:4066 , piece_id=FWSZRSF3DP2VTJQJKLGKGGZCUP62YXZEWPB5IMIKDU4QHRY46ZJA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490197063 (444h51m30.373610693s) [noise=true , address=104.36.82.34:4551 , piece_id=455MQQEMSNHTSZNEWU37DAFK4JNKKZDDWSUXFMTBOCQJ443PWWXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490604300 (444h51m30.374017929s) [noise=true , address=104.36.82.40:4962 , piece_id=JHVGP44L2LBPL234MI6VSJZYZ6G6C6VIOYGL6ZDIQSFIIX6NJ2UQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491042169 (444h51m30.374455796s) [noise=true , address=77.90.31.21:4884 , piece_id=IPGAG6G43WNCIM4E5RIHACCS4R35Q2ZZV4POIEBFG4I6D37IM72A, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491492265 (444h51m30.374905902s) [noise=true , address=104.36.82.17:4336 , piece_id=KCHNTU5TWK4X63FTNR5JTDIFGQOJXDHHCVAVU76YMS4VIAFWRUWA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491866085 (444h51m30.375279717s) [noise=true , address=45.91.95.4:4019 , piece_id=JBBJKCAXRB2GUXBL6ZJOS3JMJHR3WDDEDQ2LB36BMHO7SPZ6HKBQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.492126748 (444h51m30.375540385s) [noise=true , address=77.90.31.21:4895 , piece_id=IWTSGOW7UNMAJWC3QO3OZWFBUJNWOBTXSCQ6MMMTNUR3BNBSKTXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.492565072 (444h51m30.375978713s) [noise=true , address=104.36.82.18:4374 , piece_id=VJ5O3X2U36FH5W6N2FO7ORNLXW3KNF5QT7H355WQZL2I632HUEAA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.493333026 (444h51m30.376746662s) [noise=true , address=77.90.31.17:4788 , piece_id=TIYZBFZ3JTRNBOFAIWEJWYBMDAOJZYPQ3MWWNUYBXPHQ56VY73WQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.493702567 (444h51m30.377116204s) [noise=true , address=77.90.31.14:4692 , piece_id=WQQ7SNSTHOP4TSKRDDM76VYWJELUWOCH6GSE6XSUE4NVAIMHMPHQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.494079587 (444h51m30.377493219s) [noise=true , address=79.127.220.101:4057 , piece_id=QXEVOIUOJRDWTL7FVGAR22HIN4ZKSORAK2BARAQIRCS33BXX5X6Q, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.494555851 (444h51m30.377969484s) [noise=true , address=77.90.31.11:4571 , piece_id=LFRXPKHCSS65V6AE6P3PFDBEZ5DIKY3RFGR4GF5UWPV6P6BHPCZQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:31.011808775 (444h51m30.895222393s) [upload=355744]: Next returned nil segment Jul 11 22:02:31.011828319 (444h51m30.895241945s) [upload=355744]: Waiting for error group managing segments... Jul 11 22:02:31.185708662 (444h51m31.069122287s) [noise=true , address=77.90.31.11:4571 , piece_id=LFRXPKHCSS65V6AE6P3PFDBEZ5DIKY3RFGR4GF5UWPV6P6BHPCZQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.203802874 (444h51m31.087216495s) [noise=true , address=77.90.31.12:4617 , piece_id=WZWXQJGB4MTW4JN44NY5NQCXOWZQSGBX6IIBF56TQKB3UFQ7JEJA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.205879153 (444h51m31.089292766s) [noise=true , address=77.90.31.16:4755 , piece_id=KI32C4HB2RCC4EDMM2B6UW6OUPSMAKU5GOA5ANIXTJQD3WLQLXBQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.219680112 (444h51m31.103093739s) [noise=true , address=45.91.116.4:4966 , piece_id=K7JXTKTY4UJSCMJUOSWFWHIIZBNCXSP3N5K3INQDF77HDAKHRGIQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.222479897 (444h51m31.105893526s) [noise=true , address=77.90.31.14:4692 , piece_id=WQQ7SNSTHOP4TSKRDDM76VYWJELUWOCH6GSE6XSUE4NVAIMHMPHQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.230662544 (444h51m31.114076159s) [noise=true , address=77.90.31.21:4895 , piece_id=IWTSGOW7UNMAJWC3QO3OZWFBUJNWOBTXSCQ6MMMTNUR3BNBSKTXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.230778357 (444h51m31.114191976s) [noise=true , address=77.90.31.21:4884 , piece_id=IPGAG6G43WNCIM4E5RIHACCS4R35Q2ZZV4POIEBFG4I6D37IM72A, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.334409626 (444h51m31.217823263s) [noise=true , address=79.127.220.101:4066 , piece_id=FWSZRSF3DP2VTJQJKLGKGGZCUP62YXZEWPB5IMIKDU4QHRY46ZJA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.339502902 (444h51m31.222916533s) [noise=true , address=77.90.31.17:4788 , piece_id=TIYZBFZ3JTRNBOFAIWEJWYBMDAOJZYPQ3MWWNUYBXPHQ56VY73WQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.388877737 (444h51m31.272291372s) [noise=true , address=45.91.95.2:4016 , piece_id=OB4WV3YWOQNVUQIUAKFKDS7NF7FX54UEPBZVNNEUAORZUFE2FTXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.389126192 (444h51m31.272539822s) [noise=true , address=45.91.95.5:4002 , piece_id=3IZ3MAVFGNZO2C7LPTOY2FTIC6R2DC4OJY6XBJVWMPRE63B4DUEQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.389800397 (444h51m31.273214027s) [noise=true , address=79.127.220.101:4057 , piece_id=QXEVOIUOJRDWTL7FVGAR22HIN4ZKSORAK2BARAQIRCS33BXX5X6Q, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.426833520 (444h51m31.31024715s) [noise=true , address=45.91.95.4:4019 , piece_id=JBBJKCAXRB2GUXBL6ZJOS3JMJHR3WDDEDQ2LB36BMHO7SPZ6HKBQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:03:30.023868435 (444h52m29.907282053s) [noise=true , address=104.36.82.50:4031 , piece_id=ZTD6S46WOAD2BKVVJ3RCHX55RGJNITAVPZ3ENGQBAQB3OPLW6C2A, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:03:30.024114203 (444h52m29.90752783s) [noise=true , address=104.36.82.18:4374 , piece_id=VJ5O3X2U36FH5W6N2FO7ORNLXW3KNF5QT7H355WQZL2I632HUEAA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:1pbWgKsDigGSwnWtNHwsJC3srt45YiUhswf2hv9gbn2LSCLkSH, address:104.36.82.18:4374): protocol: write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection; write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection Jul 11 22:03:30.024186106 (444h52m29.907599732s) [noise=true , address=104.36.82.40:4962 , piece_id=JHVGP44L2LBPL234MI6VSJZYZ6G6C6VIOYGL6ZDIQSFIIX6NJ2UQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:13va7LjDtQV2ob7mrUaJUXc64HmZG3svoDaErmypjYHVAcw1o3, address:104.36.82.40:4962): protocol: write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection; write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection Jul 11 22:03:30.024207244 (444h52m29.907620871s) [noise=true , address=104.36.82.34:4551 , piece_id=455MQQEMSNHTSZNEWU37DAFK4JNKKZDDWSUXFMTBOCQJ443PWWXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:12vR5WRMGoYyzYDZzwELFw6kLG6vtKsUeFM9kUhgZDLGKB6vcXs, address:104.36.82.34:4551): protocol: write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection; write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection Jul 11 22:03:30.024253975 (444h52m29.907667602s) [noise=true , address=104.36.82.17:4336 , piece_id=KCHNTU5TWK4X63FTNR5JTDIFGQOJXDHHCVAVU76YMS4VIAFWRUWA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:1R66QktZdYxwpsjALURfq9eEvqNG2tfMvBjC5DnackVppZdGpT, address:104.36.82.17:4336): protocol: write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection; write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection Jul 11 22:03:30.024291452 (444h52m29.907705088s) [seg_pos={0 0}, upload=355744]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:03:30.137783063 (444h52m30.021196699s) [upload=355744]: Done uploading segments... Jul 11 22:03:30.137819589 (444h52m30.021233226s) [upload=355744]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 9.236809975s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 8.55313247s FAIL (2025-07-11 22:04:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4KA2CWK37V5LCFHGRPYKVBDXRYQZ4LUTMD6ZBGLELUQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224183415 (444h52m26.107597051s) [noise=true , address=77.90.31.18:4806 , piece_id=LFNM3KVOSZMB2ZOO23KEDHLP5ZY3XJLF7ZINRWGV6CURDXHEH35Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224517315 (444h52m26.107930946s) [noise=true , address=104.36.82.9:4005 , piece_id=5UDZ7TZ53Q455F7BNK6DLI3A5ONUFOWLABJPV5EB2B72CP3FJL2A, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224812761 (444h52m26.10822639s) [noise=true , address=104.36.82.15:4259 , piece_id=UXFXYHBT24C3U25JZVCOEKLBPU5VRGCPX3SDNRJCHTHKLEAULUZA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225181978 (444h52m26.108595613s) [noise=true , address=104.36.82.14:4222 , piece_id=BUC5EAICHZNODEW5MNAM26KEMQBBVVGIH2P5W7WQSEVNFSB4ZS2A, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225508695 (444h52m26.108922325s) [noise=true , address=104.36.82.12:4151 , piece_id=B3CTCC43VMTYQQHMSKFAGTT3EYMXB244IHKDNEE6F4VDJ2XBGBXA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225792461 (444h52m26.109206097s) [noise=true , address=77.90.31.14:4663 , piece_id=SH7HEPZEDQ4RX7IQMECDVTNIVIKT2NX75IPZDLQHCEFONH3MXB4Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226172056 (444h52m26.109585688s) [noise=true , address=104.36.82.7:4912 , piece_id=3QWQCGAD4ISSIHW5TQG7QXSNHW3M7J6XBQQSRH3DPTBAZOKREGPA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226481110 (444h52m26.10989474s) [noise=true , address=104.36.82.10:4078 , piece_id=QGQ4NH5ZMYFPDIN4MSDEF4SIUZBZ3JCOFKLZ7WWLZEKVZK3QCENQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226831712 (444h52m26.110245341s) [noise=true , address=104.36.82.8:4971 , piece_id=EIVPU6EWWKJSRYSMWACCY6SMFD27KFBG7DNWFD6M7WCJROUWIMIQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.227131111 (444h52m26.11054474s) [noise=true , address=45.91.116.4:4939 , piece_id=7R4FHPHRZAPYV6OXPKA7IDVLCOVWZAZMAXKHBVY2NL57OE2QCZ4Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:27.302994623 (444h52m27.186408257s) [upload=355755]: Next returned nil segment Jul 11 22:03:27.303018528 (444h52m27.186432164s) [upload=355755]: Waiting for error group managing segments... Jul 11 22:03:27.876648904 (444h52m27.760062523s) [noise=true , address=77.90.31.18:4806 , piece_id=LFNM3KVOSZMB2ZOO23KEDHLP5ZY3XJLF7ZINRWGV6CURDXHEH35Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:28.609717180 (444h52m28.493130792s) [noise=true , address=45.91.116.4:4939 , piece_id=7R4FHPHRZAPYV6OXPKA7IDVLCOVWZAZMAXKHBVY2NL57OE2QCZ4Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:31.013638539 (444h52m30.897052169s) [noise=true , address=77.90.31.14:4663 , piece_id=SH7HEPZEDQ4RX7IQMECDVTNIVIKT2NX75IPZDLQHCEFONH3MXB4Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:31.108297749 (444h52m30.991711367s) [noise=true , address=45.91.116.14:4190 , piece_id=TNIBGC6OIB5KS325JUH35L2YJYFOO4HD4XNOQX55TKQR5CRYUUHQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:33.471534051 (444h52m33.35494767s) [noise=true , address=45.91.116.11:4117 , piece_id=DIDIAZVBKR3SEC7QANMXIR6XUGWXDQQUHM6WA7XTHEYF7OLZJIYQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:37.297262395 (444h52m37.180676021s) [noise=true , address=45.91.116.14:4189 , piece_id=S2N5INQUI4KA2CWK37V5LCFHGRPYKVBDXRYQZ4LUTMD6ZBGLELUQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:55.824710938 (444h52m55.708124551s) [noise=true , address=104.36.82.8:4971 , piece_id=EIVPU6EWWKJSRYSMWACCY6SMFD27KFBG7DNWFD6M7WCJROUWIMIQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:03:55.825086456 (444h52m55.708500072s) [noise=true , address=104.36.82.9:4006 , piece_id=4FGZSUZ54DHILWD7SFWNRXKQJJG6UKRTKCYYAOSYY5Q4IVTPW6JA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection; write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection Jul 11 22:03:55.825165541 (444h52m55.708579153s) [noise=true , address=104.36.82.15:4259 , piece_id=UXFXYHBT24C3U25JZVCOEKLBPU5VRGCPX3SDNRJCHTHKLEAULUZA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection; write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection Jul 11 22:03:55.825228546 (444h52m55.708642158s) [noise=true , address=104.36.82.14:4222 , piece_id=BUC5EAICHZNODEW5MNAM26KEMQBBVVGIH2P5W7WQSEVNFSB4ZS2A, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection; write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection Jul 11 22:03:55.825237626 (444h52m55.708651277s) [noise=true , address=104.36.82.10:4078 , piece_id=QGQ4NH5ZMYFPDIN4MSDEF4SIUZBZ3JCOFKLZ7WWLZEKVZK3QCENQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection Jul 11 22:03:55.825297732 (444h52m55.708711344s) [noise=true , address=104.36.82.16:4293 , piece_id=VCINGGXB7VIPIQTGNM5II2EISTTY32OACYGGQZZ7VKKZZZHXBPTA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection; write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection Jul 11 22:03:55.825363916 (444h52m55.708777529s) [noise=true , address=104.36.82.9:4005 , piece_id=5UDZ7TZ53Q455F7BNK6DLI3A5ONUFOWLABJPV5EB2B72CP3FJL2A, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection; write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection Jul 11 22:03:55.825421328 (444h52m55.708834939s) [noise=true , address=104.36.82.7:4912 , piece_id=3QWQCGAD4ISSIHW5TQG7QXSNHW3M7J6XBQQSRH3DPTBAZOKREGPA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection; write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection Jul 11 22:03:55.825463724 (444h52m55.708877335s) [noise=true , address=104.36.82.12:4151 , piece_id=B3CTCC43VMTYQQHMSKFAGTT3EYMXB244IHKDNEE6F4VDJ2XBGBXA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection; write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection Jul 11 22:03:55.825477966 (444h52m55.708891585s) [noise=true , address=104.36.82.12:4149 , piece_id=UNAE4IGDADDPLGWRJW6S3JC5AFNSH44REOGKBTV3F7G65YY7EQDQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection; write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection Jul 11 22:03:55.825533457 (444h52m55.708947087s) [seg_pos={0 0}, upload=355755]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:03:55.935680781 (444h52m55.819094399s) [upload=355755]: Done uploading segments... Jul 11 22:03:55.935707519 (444h52m55.819121146s) [upload=355755]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 8.553163356s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XX......................................................XXX.] duration: 5.451629743s FAIL (2025-07-11 22:04:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.451650108s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)