Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 173220 reports (Page 376 of 3465)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 135.388468ms remote: "tcp:15.161.34.42:443" target: https://www.storj.io/ FAIL (2025-09-20 21:10:50 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 139.59.144.22:34312->15.161.34.42:443: read: connection reset by peer Duration: 135.425473ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 113.542456ms remote: "tcp:15.161.34.42:443" target: https://www.storj.io/ FAIL (2025-09-20 21:10:17 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 139.59.144.22:34312->15.161.34.42:443: read: connection reset by peer Duration: 113.579357ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 2.428589567s remote: "tcp:15.161.34.42:443" target: https://www.storj.io/ FAIL (2025-09-20 21:09:34 UTC) orbiter warning: Get "https://www.storj.io/": read tcp 139.59.144.22:34312->15.161.34.42:443: read: connection reset by peer Duration: 2.428622331s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.799008374s FAIL (2025-09-19 21:07:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.799038041s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.723400254s FAIL (2025-09-19 21:06:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.723440209s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.709105232s FAIL (2025-09-19 21:05:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.709206461s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..X........................................................] duration: 2.866579003s FAIL (2025-09-19 21:04:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.866610361s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.716731222s FAIL (2025-09-19 21:04:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.716757826s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.836580835s FAIL (2025-09-19 21:04:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.836609648s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 2.803550736s FAIL (2025-09-19 21:03:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.80357589s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..X.......................................................] duration: 2.47197076s FAIL (2025-09-19 21:03:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.472010124s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.X..X......................................................] duration: 2.495327259s FAIL (2025-09-19 21:03:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.495379341s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 2.865633356s FAIL (2025-09-19 21:03:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.865725325s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X..XX.......................................................] duration: 2.808523476s FAIL (2025-09-19 21:02:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.809295993s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XX.X..X.....................................................] duration: 2.795418898s FAIL (2025-09-19 21:02:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.795455053s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XX..XX......................................................] duration: 2.677532758s FAIL (2025-09-19 21:01:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.677565117s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XX.X..X....................................................] duration: 3.102706029s FAIL (2025-09-19 21:01:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 3.102744935s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XX..XX.....................................................] duration: 2.644374462s FAIL (2025-09-19 21:00:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.644475525s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.XX.X..X...................................................] duration: 2.536523118s FAIL (2025-09-19 21:00:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.536580492s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XX..XX....................................................] duration: 2.672848926s FAIL (2025-09-19 20:59:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.672876734s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XX.XX.X..X..................................................] duration: 2.612844757s FAIL (2025-09-19 20:59:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.612879239s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...XX..XX...................................................] duration: 2.669662443s FAIL (2025-09-19 20:59:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.669689061s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....XX..XX..................................................] duration: 3.159999462s FAIL (2025-09-19 20:59:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.160104886s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XXX.XX.X..X.................................................] duration: 2.558339175s FAIL (2025-09-19 20:58:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.558367193s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X....XX..XX.................................................] duration: 2.710159298s FAIL (2025-09-19 20:58:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.710261633s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXXX.XX.X..X................................................] duration: 2.419258974s FAIL (2025-09-19 20:58:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.419287501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXXXX.XX.X..X...............................................] duration: 2.636901948s FAIL (2025-09-19 20:58:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.636993162s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXXXX.XX.X..X..............................................] duration: 2.701942853s FAIL (2025-09-19 20:57:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.701967567s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XX....XX..XX................................................] duration: 2.978640421s FAIL (2025-09-19 20:57:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.978678772s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XXX....XX..XX...............................................] duration: 2.748275708s FAIL (2025-09-19 20:56:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.748301252s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXXXXX.XX.X..X.............................................] duration: 2.547538752s FAIL (2025-09-19 20:56:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.547583027s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [XXXXXXXX.XX.X..X............................................] duration: 2.684000541s FAIL (2025-09-19 20:56:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.68403384s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXXX....XX..XX..............................................] duration: 2.813678444s FAIL (2025-09-19 20:56:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.813704019s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXXXX....XX..XX.............................................] duration: 3.035687356s FAIL (2025-09-19 20:55:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.035719268s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [XXXXXXXXX.XX.X..X...........................................] duration: 2.404831046s FAIL (2025-09-19 20:55:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.404854285s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXXXX....XX..XX............................................] duration: 2.714741831s FAIL (2025-09-19 20:54:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.714771105s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.XXXXXXXXX.XX.X..X..........................................] duration: 2.680623982s FAIL (2025-09-19 20:54:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.680663077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..XXXXXXXXX.XX.X..X.........................................] duration: 2.574333168s FAIL (2025-09-19 20:53:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.57436511s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXXXXX....XX..XX...........................................] duration: 3.008397914s FAIL (2025-09-19 20:53:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.008424328s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [XXXXXXXX....XX..XX..........................................] duration: 2.698285036s FAIL (2025-09-19 20:53:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.698309625s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [XXXXXXXXX....XX..XX.........................................] duration: 2.754656034s FAIL (2025-09-19 20:52:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.75468234s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X..XXXXXXXXX.XX.X..X........................................] duration: 2.582392813s FAIL (2025-09-19 20:52:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.582427241s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.XXXXXXXXX....XX..XX........................................] duration: 3.001706492s FAIL (2025-09-19 20:52:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.001730234s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.X..XXXXXXXXX.XX.X..X.......................................] duration: 2.735323004s FAIL (2025-09-19 20:52:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.735375761s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..XXXXXXXXX....XX..XX.......................................] duration: 2.813912673s FAIL (2025-09-19 20:51:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.813951305s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X..XXXXXXXXX.XX.X..X......................................] duration: 2.392696525s FAIL (2025-09-19 20:51:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.392786205s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...X..XXXXXXXXX.XX.X..X.....................................] duration: 2.772596919s FAIL (2025-09-19 20:51:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.772634638s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...XXXXXXXXX....XX..XX......................................] duration: 2.67327089s FAIL (2025-09-19 20:50:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.673307301s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [....X..XXXXXXXXX.XX.X..X....................................] duration: 2.56005485s FAIL (2025-09-19 20:50:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.56009123s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [....XXXXXXXXX....XX..XX.....................................] duration: 2.799104132s FAIL (2025-09-19 20:50:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.799171484s edge gateway us-select-1

1
...
374
375
376
377
378
...
3465