Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.787703s FAIL (2025-09-18 06:59:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.787752637s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.944557663s FAIL (2025-09-18 06:59:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.944606941s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.651299566s FAIL (2025-09-18 06:59:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.65133919s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.747147725s FAIL (2025-09-18 06:59:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.747192649s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.706404977s FAIL (2025-09-18 06:58:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.706431468s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.688836219s FAIL (2025-09-18 06:58:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.688863157s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.891966466s FAIL (2025-09-18 06:58:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.892007937s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 3.083832878s FAIL (2025-09-18 06:58:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.083869519s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.415640922s FAIL (2025-09-18 06:57:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.415678922s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 3.032348769s FAIL (2025-09-18 06:57:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.032383853s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.458144884s FAIL (2025-09-18 06:57:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.458197563s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 3.385438426s FAIL (2025-09-18 06:57:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.385464932s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.917848693s FAIL (2025-09-18 06:57:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.918030134s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.749505638s FAIL (2025-09-18 06:57:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.749634945s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.592791983s FAIL (2025-09-18 06:56:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.592819796s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.459880808s FAIL (2025-09-18 06:56:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.460191759s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 3.24112815s FAIL (2025-09-18 06:56:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.241164866s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.95079859s FAIL (2025-09-18 06:55:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.950827764s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.403952267s FAIL (2025-09-18 06:55:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.404082932s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.739730181s FAIL (2025-09-18 06:55:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.739779445s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.777711152s FAIL (2025-09-18 06:55:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.777750267s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.749206s FAIL (2025-09-18 06:54:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.749310702s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.370563874s FAIL (2025-09-18 06:54:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.370599639s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.856724555s FAIL (2025-09-18 06:54:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.85682294s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.411180603s FAIL (2025-09-18 06:54:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.411269469s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.913027059s FAIL (2025-09-18 06:54:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.913067894s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.999513127s FAIL (2025-09-18 06:54:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.999559529s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 3.05850318s FAIL (2025-09-18 06:53:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.058538169s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.591622555s FAIL (2025-09-18 06:53:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.591655956s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.805859279s FAIL (2025-09-18 06:53:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.8058832s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.701673169s FAIL (2025-09-18 06:52:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.701696261s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 3.325234307s FAIL (2025-09-18 06:52:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.325320536s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.502135411s FAIL (2025-09-18 06:52:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.502173798s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 3.083882167s FAIL (2025-09-18 06:52:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.083984996s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.376239227s FAIL (2025-09-18 06:52:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.376322957s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.974754752s FAIL (2025-09-18 06:52:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.974885762s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.933901077s FAIL (2025-09-18 06:51:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.9339244s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.504285425s FAIL (2025-09-18 06:51:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.504319845s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.652709755s FAIL (2025-09-18 06:51:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.652734161s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.972461264s FAIL (2025-09-18 06:51:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.972523435s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.753608776s FAIL (2025-09-18 06:50:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.753671194s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.825158999s FAIL (2025-09-18 06:50:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.8252539s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.677747502s FAIL (2025-09-18 06:50:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.677780496s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.76847174s FAIL (2025-09-18 06:49:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.768498444s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.742612671s FAIL (2025-09-18 06:49:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.742639559s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.417096282s FAIL (2025-09-18 06:49:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 2.417130242s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 3.083099205s FAIL (2025-09-18 06:48:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== .10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606164283 (472h12m51.098596735s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.606697593 (472h12m51.099130037s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608051611 (472h12m51.100484056s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.608678493 (472h12m51.101110944s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.609893324 (472h12m51.102325779s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.610954207 (472h12m51.103386656s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611458361 (472h12m51.103890813s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.611966832 (472h12m51.104399283s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.615074539 (472h12m51.107506985s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.617371261 (472h12m51.109803715s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.618213880 (472h12m51.110646334s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:30.619133336 (472h12m51.111565791s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Uploading piece... Sep 18 06:20:31.285536041 (472h12m51.77796849s) [upload=415363]: Next returned nil segment Sep 18 06:20:31.285571077 (472h12m51.778003528s) [upload=415363]: Waiting for error group managing segments... Sep 18 06:20:31.397318156 (472h12m51.889750585s) [noise=true , address=145.79.227.3:4150 , piece_id=6EBIJLTHIJKVODLIBPRYY2ZQPW6NJETKPR4JTZBXYC4OCTIKNF4Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.409603457 (472h12m51.902035902s) [noise=true , address=145.79.227.13:4802 , piece_id=XDUCHMCSIFXZUN7HIZCOUTBH3WGAUOYV3NRI7JNXA4FHDFQAYTXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:20:31.412562303 (472h12m51.904994735s) [noise=true , address=145.79.227.19:4077 , piece_id=DAGFGYSY6GRWVDGDQHH2XJXUSINUWRNPCQYZSBEBDTWNXDZ3DIGA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: <nil> Sep 18 06:21:00.171009301 (472h13m20.663441758s) [noise=true , address=45.91.116.12:4226 , piece_id=7NKKWSTZ3WTW4NA7VTRWSSPSNBLXW3NJBOCEDT34NKYSGVD7UFCA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:54302->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.171668056 (472h13m20.664100511s) [noise=true , address=45.91.116.10:4081 , piece_id=RHWZE4VNRGE4GRRKSUE2JZGLK2GSC6VJI5KAUVCSV2L3OZBDQLXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 139.59.144.22:37718->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172033515 (472h13m20.664465965s) [noise=true , address=77.90.31.16:4734 , piece_id=HGNBJ7J3C5S5PIW4IEJGM6CYNZVJNI7XVHV3CR7GWE5QGUR3EYEA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172061093 (472h13m20.664493549s) [noise=true , address=45.91.116.5:4297 , piece_id=4XPFFRMWOX6NW2ZB6AVJ2NZH2MPSQLW67SAMJFT7BCYOEQKL4SOA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: rpc: dial tcp 45.91.116.5:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172531910 (472h13m20.664964367s) [noise=true , address=45.91.116.14:4191 , piece_id=AB4SNVFELNE6JQPTWQ7X5NQXDIMUJJXSJZDL4RRR7OY6XXPOR4CA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172634427 (472h13m20.665066857s) [noise=true , address=45.91.116.7:4010 , piece_id=4CKR6F2YD2OAGWVRRAWMMQ7EQUH6NUWOQVYQHGCT4EYMFWGXHHXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 139.59.144.22:37268->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172651500 (472h13m20.66508395s) [noise=true , address=45.91.116.13:4155 , piece_id=HN7PMJACV7KXH7PSOGIF55T7KV6AHKVBLTDFLFUKM7WTFBEW7N4A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 139.59.144.22:45276->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172708335 (472h13m20.665140789s) [noise=true , address=77.90.31.14:4662 , piece_id=6HOFUMZOGMABSV2UKSU4B3YUQCW33FIFB2Y5AKMXQXRT3SNMDDLA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: read tcp 139.59.144.22:59692->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172770932 (472h13m20.665203365s) [noise=true , address=45.91.116.4:4939 , piece_id=IQCTRATIFNKF7YEGNYP52PUXCMT2N23AFVSX37BBW42IK6MXTBXA, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:43042->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172832116 (472h13m20.665264567s) [noise=true , address=45.91.116.6:4973 , piece_id=R2H5RBFIX4BUUKWJGMPQPEM5EDQ3OLL6MFVTXFUF7RQTN57BVD7A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:49606->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172871254 (472h13m20.665303689s) [noise=true , address=45.91.116.11:4118 , piece_id=2V6WZWKGO6SNLB2UR5Q5TP5VKBPB3JBJ2HUMXEAAWICBN472P47A, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55146->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172885844 (472h13m20.665318284s) [noise=true , address=45.91.116.13:4154 , piece_id=3W2YWDE54STQE25JSBSAFS2XRFELZKHSZXVQEEHAWJ6LNJHVJGFQ, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 139.59.144.22:59448->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172898443 (472h13m20.665330883s) [noise=true , address=45.91.116.11:4119 , piece_id=2BUVZ3ZMQCR47NJZFHA7DF53PWKHKLHGTSHBTRSR5EZHWKXZX67Q, seg_pos={0 0}, upload=415363]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 139.59.144.22:55382->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 18 06:21:00.172972913 (472h13m20.665405371s) [seg_pos={0 0}, upload=415363]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 18 06:21:00.285796348 (472h13m20.778228783s) [upload=415363]: Done uploading segments... Sep 18 06:21:00.285852252 (472h13m20.778284705s) [upload=415363]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 3.083132063s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.904394586s FAIL (2025-09-18 06:48:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.904442933s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.828233008s FAIL (2025-09-18 06:48:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.828264691s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 3.002371033s FAIL (2025-09-18 06:47:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.002397408s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)