Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.511975656s FAIL (2025-09-18 06:36:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.512026925s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.985437198s FAIL (2025-09-18 06:36:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.985520615s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.672571252s FAIL (2025-09-18 06:35:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.672608599s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.88361926s FAIL (2025-09-18 06:35:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.883658961s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 3.304885951s FAIL (2025-09-18 06:35:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.304920362s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 3.173000378s FAIL (2025-09-18 06:34:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.173149135s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.896255577s FAIL (2025-09-18 06:34:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.89631435s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.380612999s FAIL (2025-09-18 06:34:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.380650818s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.978353877s FAIL (2025-09-18 06:34:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.978394318s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 3.139824725s FAIL (2025-09-18 06:33:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.139857632s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.63004946s FAIL (2025-09-18 06:33:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.630075828s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.601751362s FAIL (2025-09-18 06:33:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.601787384s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.637559994s FAIL (2025-09-18 06:33:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.637620686s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.808623668s FAIL (2025-09-18 06:33:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.808679122s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 3.246440042s FAIL (2025-09-18 06:32:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.246469421s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.460211266s FAIL (2025-09-18 06:32:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.460247798s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.443166908s FAIL (2025-09-18 06:32:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.443203047s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.610139027s FAIL (2025-09-18 06:31:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.610172013s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.785084172s FAIL (2025-09-18 06:31:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.78511713s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.441191414s FAIL (2025-09-18 06:31:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.441230428s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.823902114s FAIL (2025-09-18 06:31:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.823935055s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 3.187097374s FAIL (2025-09-18 06:31:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.187182891s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 3.11407594s FAIL (2025-09-18 06:30:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.114209516s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.530753138s FAIL (2025-09-18 06:30:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.530854321s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.900137002s FAIL (2025-09-18 06:30:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.900164767s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 3.025440829s FAIL (2025-09-18 06:30:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.025516484s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.5778127s FAIL (2025-09-18 06:30:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.577865601s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.87205494s FAIL (2025-09-18 06:30:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.872126448s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.703993754s FAIL (2025-09-18 06:29:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.704028203s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.673090899s FAIL (2025-09-18 06:29:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.673119887s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.751887212s FAIL (2025-09-18 06:29:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.751997396s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.827808893s FAIL (2025-09-18 06:28:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.827865293s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.352555953s FAIL (2025-09-18 06:28:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.352586623s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.817262511s FAIL (2025-09-18 06:28:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.817274852s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.874872446s FAIL (2025-09-18 06:28:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.874906897s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.873144832s FAIL (2025-09-18 06:28:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.873179156s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.63686866s FAIL (2025-09-18 06:27:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.636918776s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.877809049s FAIL (2025-09-18 06:27:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.877977296s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 3.325649874s FAIL (2025-09-18 06:27:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.3256785s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.85794702s FAIL (2025-09-18 06:26:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.857969546s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.78361654s FAIL (2025-09-18 06:26:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.783649569s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.544941535s FAIL (2025-09-18 06:26:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.544975356s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.766726953s FAIL (2025-09-18 06:26:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.766753343s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.93481005s FAIL (2025-09-18 06:26:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.934849084s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.845366405s FAIL (2025-09-18 06:25:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.845394346s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.442422129s FAIL (2025-09-18 06:25:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.442474668s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 3.293601147s FAIL (2025-09-18 06:25:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.29363219s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.518117939s FAIL (2025-09-18 06:24:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.518212062s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.771757681s FAIL (2025-09-18 06:24:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.771795311s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.376268365s FAIL (2025-09-18 06:24:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.376376527s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)