Orbiter History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 392.778208ms remote: "tcp:15.161.34.42:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-11 23:53:24 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 139.59.144.22:39346->15.161.34.42:443: read: connection reset by peer Duration: 392.802026ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 243.868707ms remote: "tcp:15.161.34.42:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-11 23:53:07 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 139.59.144.22:39346->15.161.34.42:443: read: connection reset by peer Duration: 243.894493ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 94.050463ms remote: "tcp:35.152.117.67:443" target: https://www.storj.io/ FAIL (2025-07-11 23:52:36 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 139.59.144.22:52096->35.152.117.67:443: read: connection reset by peer Duration: 94.079265ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 207.362616ms remote: "tcp:15.161.34.42:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-11 23:52:12 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 139.59.144.22:39346->15.161.34.42:443: read: connection reset by peer Duration: 207.413119ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 112.385999ms remote: "tcp:35.152.117.67:443" target: https://www.storj.io/ FAIL (2025-07-11 23:51:46 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 139.59.144.22:52096->35.152.117.67:443: read: connection reset by peer Duration: 112.4139ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 233.222276ms remote: "tcp:15.161.34.42:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-11 23:51:39 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 139.59.144.22:39346->15.161.34.42:443: read: connection reset by peer Duration: 233.246469ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 8.332068009s remote: "tcp:15.161.34.42:443" target: https://www.storj.io/dcs-satellites FAIL (2025-07-11 23:51:05 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp 139.59.144.22:39346->15.161.34.42:443: read: connection reset by peer Duration: 8.332090274s core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 3.56735268s remote: "tcp:35.152.117.67:443" target: https://www.storj.io/ FAIL (2025-07-11 23:50:50 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 139.59.144.22:52096->35.152.117.67:443: read: connection reset by peer Duration: 3.567377379s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.48729137s FAIL (2025-07-11 22:52:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.487342283s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.673718121s FAIL (2025-07-11 22:51:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.673787825s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.845026802s FAIL (2025-07-11 22:51:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.845054821s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.441992666s FAIL (2025-07-11 22:51:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.442025892s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 2.753105854s FAIL (2025-07-11 22:50:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.753130992s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.489313428s FAIL (2025-07-11 22:50:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.489343385s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 2.465053617s FAIL (2025-07-11 22:50:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.465140299s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.521210555s FAIL (2025-07-11 22:49:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.521258653s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 2.808950842s FAIL (2025-07-11 22:49:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.808972778s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XXX........................................................] duration: 2.464097603s FAIL (2025-07-11 22:49:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.464130466s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 2.536625939s FAIL (2025-07-11 22:49:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.536672689s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXXXX.......................................................] duration: 2.652316784s FAIL (2025-07-11 22:49:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.652340645s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.XXXXX......................................................] duration: 2.541995621s FAIL (2025-07-11 22:48:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.542024656s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.XXX.......................................................] duration: 2.423717148s FAIL (2025-07-11 22:48:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.423819888s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 2.709917744s FAIL (2025-07-11 22:48:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.709946883s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [X..XX.......................................................] duration: 2.721393887s FAIL (2025-07-11 22:48:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.721427644s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.XXXXX.....................................................] duration: 2.85737066s FAIL (2025-07-11 22:48:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.857469565s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XX.XXX......................................................] duration: 2.42245496s FAIL (2025-07-11 22:47:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.422501002s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XXX.XXX.....................................................] duration: 2.554638302s FAIL (2025-07-11 22:47:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.554667624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [XX..XX......................................................] duration: 2.684465848s FAIL (2025-07-11 22:47:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.684495508s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.XXX.XXX....................................................] duration: 2.42973237s FAIL (2025-07-11 22:46:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.429782679s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XX.XXXXX....................................................] duration: 2.736766088s FAIL (2025-07-11 22:46:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.736797599s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.XX..XX.....................................................] duration: 2.815627598s FAIL (2025-07-11 22:46:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.815659548s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXX.XXXXX...................................................] duration: 2.524410577s FAIL (2025-07-11 22:46:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.524436323s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..XXX.XXX...................................................] duration: 2.438906194s FAIL (2025-07-11 22:46:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.438945482s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.XXX.XXXXX..................................................] duration: 2.625448471s FAIL (2025-07-11 22:45:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.625487557s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..XX..XX....................................................] duration: 2.500025715s FAIL (2025-07-11 22:45:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.500058076s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...XXX.XXX..................................................] duration: 2.482995421s FAIL (2025-07-11 22:45:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.48303352s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...XX..XX...................................................] duration: 2.688351565s FAIL (2025-07-11 22:44:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.688386667s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..XXX.XXXXX.................................................] duration: 2.869088386s FAIL (2025-07-11 22:44:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.869118417s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....XXX.XXX.................................................] duration: 5.915876075s FAIL (2025-07-11 22:44:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.915910422s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....XXX.XXX................................................] duration: 3.613213734s FAIL (2025-07-11 22:44:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 3.613255124s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....XX..XX..................................................] duration: 2.579504387s FAIL (2025-07-11 22:43:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.57954373s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...XXX.XXXXX................................................] duration: 2.618064888s FAIL (2025-07-11 22:43:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.618093383s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......XXX.XXX...............................................] duration: 2.51166865s FAIL (2025-07-11 22:43:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.511721151s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.....XX..XX.................................................] duration: 2.746316972s FAIL (2025-07-11 22:43:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.74635039s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......XXX.XXX..............................................] duration: 2.401918494s FAIL (2025-07-11 22:42:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.401955131s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......XX..XX................................................] duration: 2.566110268s FAIL (2025-07-11 22:42:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.566146858s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....XXX.XXXXX...............................................] duration: 2.629710964s FAIL (2025-07-11 22:42:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.629737409s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.....XXX.XXXXX..............................................] duration: 2.712650312s FAIL (2025-07-11 22:42:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.712681501s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......XX..XX...............................................] duration: 2.632694386s FAIL (2025-07-11 22:41:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.632729038s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........XXX.XXX.............................................] duration: 2.334297432s FAIL (2025-07-11 22:41:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 05.834196117 (444h56m5.717609755s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834843197 (444h56m5.718256835s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.834982650 (444h56m5.718396288s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835872519 (444h56m5.719286158s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.835927919 (444h56m5.719341551s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836802263 (444h56m5.7202159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.836874595 (444h56m5.720288233s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837685342 (444h56m5.721098983s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.837838250 (444h56m5.721251887s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838450447 (444h56m5.721864083s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.838843870 (444h56m5.722257507s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.839287996 (444h56m5.722701631s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840402981 (444h56m5.723816613s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.840797623 (444h56m5.724211252s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841281296 (444h56m5.724694929s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:05.841695089 (444h56m5.725108718s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Uploading piece... Jul 11 22:07:06.462006301 (444h56m6.345419934s) [upload=355799]: Next returned nil segment Jul 11 22:07:06.462031836 (444h56m6.345445474s) [upload=355799]: Waiting for error group managing segments... Jul 11 22:07:06.598768003 (444h56m6.48218162s) [noise=true , address=45.91.116.11:4119 , piece_id=PVHH7W43BX6YDWSTCDJIRAPMQO7NTIO5KLZC6KKK3TQCKOAICRNA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.602310096 (444h56m6.485723719s) [noise=true , address=45.91.116.8:4047 , piece_id=MVP45GE7QJ7HDS5AGQKIZF5GLSCARH2KWX4UNCC3UBITYQS7S2OQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:06.629301214 (444h56m6.512714826s) [noise=true , address=45.91.116.7:4010 , piece_id=CJUVJNAZCYXL7XDTRDJ33UMQWZ7H3MHDRTQFIRYNJQIVUNQFOQZQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.150234933 (444h56m8.033648546s) [noise=true , address=77.90.31.10:4517 , piece_id=XEBB7CHHWC2OXFHWGVBEZ2K2USUXGCU3JDUO5USFFGO6TWLBEY7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:08.496869058 (444h56m8.380282674s) [noise=true , address=77.90.31.12:4590 , piece_id=MWFJ54TQUCJMIEYVNAIKTHDTZKEODZYEA3OG5TVAP3BCIEVVOLVA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:09.251099401 (444h56m9.13451302s) [noise=true , address=77.90.31.11:4555 , piece_id=DPK4OJKTRZ6D7YFE36BBQI7BTB3FTKUGBAI7B4H3FKFCMJLPI3GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: <nil> Jul 11 22:07:35.448788537 (444h56m35.332202156s) [noise=true , address=104.36.82.12:4151 , piece_id=OWRDAGMSSWCKNQDSDTLX5RUEKUCQLLBWNWJGEHYIRN3XISUBNOBA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:54572->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448869305 (444h56m35.332282922s) [noise=true , address=104.36.82.11:4114 , piece_id=X6VGOMGCLHMJADYHPJRW3HJW2WTRDDAPVW42676EY6ONHVR6I4GA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:42960->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448926785 (444h56m35.332340401s) [noise=true , address=104.36.82.8:4972 , piece_id=H422QR7RSKFETN3EXKOGH4DCU3LYTVPJ4MCCOEEAPGNJ32TDGPYQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:33578->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.448980454 (444h56m35.332394072s) [noise=true , address=104.36.82.13:4186 , piece_id=KXX7AKAIUM3TKGJ7KBTAWOCJIWCRN5QYFLMFZNRD2K6MYSI6LC4Q, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:53804->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449012540 (444h56m35.332426159s) [noise=true , address=104.36.82.14:4222 , piece_id=LIDA4EYN65LVIYBXMQYZELR73ZJQBJF7HTPOARMYGEBHB74THGFA, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:58276->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449145824 (444h56m35.332559453s) [noise=true , address=104.36.82.9:4007 , piece_id=3AFAOFKNOT5RGIZCP7YEUKL4P4SSTN27QJQ3QNEDJPPBP6OJXOJQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46048->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449231791 (444h56m35.332645408s) [noise=true , address=104.36.82.6:4876 , piece_id=ORBK5BUTLAN2QLDPAXNPBPFZB243AOYZTFHS2JXKVXJX544VJX7A, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:50862->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449288617 (444h56m35.332702234s) [noise=true , address=104.36.82.6:4874 , piece_id=3OVKEFBZHC4O6VV5YMBOZ4P4EBJM4E3YVB23E6DARRRISEAIZ3NQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:58020->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449303227 (444h56m35.332716868s) [noise=true , address=104.36.82.11:4113 , piece_id=U7QFQEMAAIAPLPEUD446QIC3LLU6VSWCXUYB6ZF66IPT5JS73WMQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:45960->104.36.82.11:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449358869 (444h56m35.332772487s) [noise=true , address=104.36.82.4:4802 , piece_id=C5ZYYVTDYZ5RVSUZ2TEVMOQT27QKLFODDNWVUJSVWNXK55HT2RPQ, seg_pos={0 0}, upload=355799]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:36524->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:35.449385152 (444h56m35.332798781s) [seg_pos={0 0}, upload=355799]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:07:35.559959616 (444h56m35.443373245s) [upload=355799]: Done uploading segments... Jul 11 22:07:35.559989423 (444h56m35.443403049s) [upload=355799]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.334341229s core uplink us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)