Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.............................XXXXXXXXX....XX..XX............] duration: 2.772912206s FAIL (2025-09-19 20:30:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.772941954s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..............................XXXXXXXXX....XX..XX...........] duration: 2.76102807s FAIL (2025-09-19 20:29:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.761054425s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...............................X..XXXXXXXXX.XX.X..X.........] duration: 2.589473173s FAIL (2025-09-19 20:29:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.589501963s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [................................X..XXXXXXXXX.XX.X..X........] duration: 2.416348389s FAIL (2025-09-19 20:29:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.416381119s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...............................XXXXXXXXX....XX..XX..........] duration: 2.767205028s FAIL (2025-09-19 20:28:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.767635254s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.................................X..XXXXXXXXX.XX.X..X.......] duration: 2.481141335s FAIL (2025-09-19 20:28:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.481177155s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [................................XXXXXXXXX....XX..XX.........] duration: 2.659637771s FAIL (2025-09-19 20:28:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.659665706s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..................................X..XXXXXXXXX.XX.X..X......] duration: 2.567569624s FAIL (2025-09-19 20:27:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.567601595s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.................................XXXXXXXXX....XX..XX........] duration: 2.798025124s FAIL (2025-09-19 20:27:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.798110928s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..................................XXXXXXXXX....XX..XX.......] duration: 2.670928838s FAIL (2025-09-19 20:27:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.670963237s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...................................X..XXXXXXXXX.XX.X..X.....] duration: 2.570929403s FAIL (2025-09-19 20:27:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.571031852s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [....................................X..XXXXXXXXX.XX.X..X....] duration: 2.798941819s FAIL (2025-09-19 20:26:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.799000914s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...................................XXXXXXXXX....XX..XX......] duration: 2.676170565s FAIL (2025-09-19 20:26:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.67688745s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [....................................XXXXXXXXX....XX..XX.....] duration: 2.664439674s FAIL (2025-09-19 20:25:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.664474408s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.....................................X..XXXXXXXXX.XX.X..X...] duration: 2.699014127s FAIL (2025-09-19 20:25:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.699042976s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.....................................XXXXXXXXX....XX..XX....] duration: 2.793218822s FAIL (2025-09-19 20:25:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.793323625s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......................................X..XXXXXXXXX.XX.X..X..] duration: 2.428374896s FAIL (2025-09-19 20:24:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.428415977s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [......................................XXXXXXXXX....XX..XX...] duration: 2.931175983s FAIL (2025-09-19 20:24:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.931232509s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......................................XXXXXXXXX....XX..XX..] duration: 2.745649872s FAIL (2025-09-19 20:24:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.745680141s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......................................X..XXXXXXXXX.XX.X..X.] duration: 2.755680915s FAIL (2025-09-19 20:23:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.755714471s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [........................................XXXXXXXXX....XX..XX.] duration: 2.82764965s FAIL (2025-09-19 20:23:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.827692702s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........................................XXXXXXXXX....XX..XX] duration: 30.001212859s FAIL (2025-09-19 20:23:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00123904s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [........................................X..XXXXXXXXX.XX.X..X] duration: 30.109809724s FAIL (2025-09-19 20:23:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== n Sep 19 20:22:50.102662820 (510h15m10.595095264s) [seg_pos={0 0}, upload=448856]: Begin upload segment... Sep 19 20:22:50.102722619 (510h15m10.595155062s) [seg_pos={0 0}, upload=448856]: Done begin upload segment. Sep 19 20:22:50.102750551 (510h15m10.595182996s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103190782 (510h15m10.595623231s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103509646 (510h15m10.5959421s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.103836285 (510h15m10.596268729s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104244391 (510h15m10.596676842s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104576441 (510h15m10.597008886s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.104941184 (510h15m10.597373631s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105304428 (510h15m10.597736873s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.105656379 (510h15m10.598088821s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106224032 (510h15m10.598656485s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.106687764 (510h15m10.599120214s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107080197 (510h15m10.599512642s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107408891 (510h15m10.599841343s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.107777408 (510h15m10.600209851s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108151816 (510h15m10.60058426s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.108465540 (510h15m10.600897982s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Uploading piece... Sep 19 20:22:50.732883115 (510h15m11.225315545s) [upload=448856]: Next returned nil segment Sep 19 20:22:50.732893407 (510h15m11.225325849s) [upload=448856]: Waiting for error group managing segments... Sep 19 20:22:50.877517094 (510h15m11.369949523s) [noise=true , address=145.79.227.6:4257 , piece_id=XFDALDBMOKBIVAXYL5EILTGUIZ63VTI766UAV4E3TRVC5BJ72LYA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.878502870 (510h15m11.370935313s) [noise=true , address=145.79.227.11:4729 , piece_id=PUYLJ6YC7RCS4EMRECHPWU5ZP5X32QSXAPPHVHAT7E5VNSK2OMXQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.884341437 (510h15m11.376773865s) [noise=true , address=145.79.227.4:4186 , piece_id=REAFHF5U3W5SQW4DSVFDW76DMFO5GVJCLOAIDWGMM2R23DRXF4RA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.890056744 (510h15m11.382489179s) [noise=true , address=145.79.227.17:4971 , piece_id=HLB4FCJNQQT2JSIHRAZ4QE4AW4DXP55SMGW3PF5UIFMSGCOZR74A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.893057011 (510h15m11.385489439s) [noise=true , address=145.79.227.18:4005 , piece_id=GCD62MFRZVI57D5XHXKKUCMB3Z7ATSBCYD7ZKBHR3VQX3JJBTSNA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.897882530 (510h15m11.390314966s) [noise=true , address=145.79.227.15:4873 , piece_id=R6T45TIWE2TWJ2OK7ZZHMWZVNYEIV5PN5QSMUXMN73RN2YKEGFQQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:22:50.906019627 (510h15m11.398452056s) [noise=true , address=145.79.227.14:4839 , piece_id=C6P6TEA3WU3D5RLR7JIGDQTIA3IW25WJQSNCZSIPKSEPQDNFRD2Q, seg_pos={0 0}, upload=448856]: Done uploading piece. err: <nil> Sep 19 20:23:19.696847719 (510h15m40.189280165s) [noise=true , address=45.91.116.10:4081 , piece_id=J2HFTS6GURXVDX7LUFNJANXL66YDYSOHHEZX6HU7F5PLFEKFC7KQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:46728->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696918229 (510h15m40.189350671s) [noise=true , address=77.90.31.19:4842 , piece_id=3UAFAGUVXME73AQA2WG7TMIIKRNJNEEZ4LBZEX7BFZNX7NWC2NPQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:49418->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.696973359 (510h15m40.189405802s) [noise=true , address=77.90.31.10:4517 , piece_id=B52JS5VGEWC7RKBVI4DI5CFYWUEJJWS6JTG2OY4UVGFVBCPDJJIA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:44372->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697023793 (510h15m40.189456228s) [noise=true , address=45.91.116.7:4009 , piece_id=OIAJ5EU4CW44PHYNDOKMLFQQ226C6MPAMKYFHKIOESYXITQE2VHA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 139.59.144.22:51982->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697077949 (510h15m40.189510391s) [noise=true , address=45.91.116.12:4225 , piece_id=QS7LEWFVLRU3ATOPBDPX7UCQWAPATAI6VYSMNQLHZA3U6JLBTCFQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:56108->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697122743 (510h15m40.189555185s) [noise=true , address=45.91.116.14:4189 , piece_id=HV6YRMRJXXN2W6SKXQNBXYBM23275QFCZNPANEWZ7U6QD6JLYD5A, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 139.59.144.22:57576->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697151202 (510h15m40.189583638s) [noise=true , address=45.91.116.6:4974 , piece_id=MCWDOVJQTUDENKGKJ3W4GWACOLLEDHVAJLNY2OY43BHIVLMLWAGA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:59176->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697194787 (510h15m40.189627229s) [noise=true , address=45.91.116.11:4117 , piece_id=A4PAAXKY6WPJJXDEZZ4EKUO3XSPKRIEDHCBKH3FYRLY4RGQQ4ALQ, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:41286->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:19.697245542 (510h15m40.189677992s) [noise=true , address=45.91.116.14:4190 , piece_id=PKGXK4NJ2V3K2VOCMPE7IZEGXUX6TFNEJOAZU66MCBINK6BV6JAA, seg_pos={0 0}, upload=448856]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:56168->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:23:19.697276993 (510h15m40.189709442s) [seg_pos={0 0}, upload=448856]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:23:19.804537315 (510h15m40.296969749s) [upload=448856]: Done uploading segments... Sep 19 20:23:19.804563258 (510h15m40.296995693s) [upload=448856]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.109830293s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........................................XXXXXXXXX....XX..X] duration: 30.001501073s FAIL (2025-09-19 20:23:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001522544s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........................................XXXXXXXXX....XX..] duration: 2.780692426s FAIL (2025-09-19 20:21:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.780723713s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........................................X..XXXXXXXXX.XX.X..] duration: 2.436281217s FAIL (2025-09-19 20:21:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== m27.772288119s) [seg_pos={0 0}, upload=448816]: Begin upload segment... Sep 19 20:20:07.279923237 (510h12m27.772355692s) [seg_pos={0 0}, upload=448816]: Done begin upload segment. Sep 19 20:20:07.279990206 (510h12m27.772422659s) [noise=true , address=45.91.116.4:4939 , piece_id=APAW4FND2AXUZB3KTMLR64RM5PTFJ3ELCAJKOHLYWLBTZTZYT5FQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.280530093 (510h12m27.772962545s) [noise=true , address=145.79.227.4:4187 , piece_id=5DH3FZIS7T26ZYWCTF3HODHTYCEWMDF6CNZUA6MVSLJJIWIGUIGA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.280907172 (510h12m27.773339624s) [noise=true , address=145.79.227.14:4838 , piece_id=XELCYITGH37HWVBO5AHX6CFTWI5WBX2EGD6GYTBHTL4VZGDVDOJA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.281250541 (510h12m27.773682993s) [noise=true , address=145.79.227.13:4801 , piece_id=WWBMH6SBFZWJHGLBYK6Q4LHV6VACKZTPH4LQELZPVXKORUQJ56IA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.281582509 (510h12m27.774014961s) [noise=true , address=145.79.227.5:4222 , piece_id=MSAWY6N63IL5FHT5GCYAHH7HVNW7B2FJB4U5TFHY2PU7WFKRCKBQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282001174 (510h12m27.774433629s) [noise=true , address=77.90.31.19:4842 , piece_id=OLGYQUMO4Z45GY7LWWMJZEZRVROHYVRH37VYRKTRHBNMBAIFNRJA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282350878 (510h12m27.774783333s) [noise=true , address=145.79.227.7:4293 , piece_id=W7FHTBVEIEMWSVWGOFDD4DQQDRHXFEIQKLG35HBEB5WX3MHVODWA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282718781 (510h12m27.775151235s) [noise=true , address=145.79.227.16:4912 , piece_id=75ZJUOXV7CF5U6O5ZQ5RFI6NVWEPY7DW3DETBSRXHFBYDU3RHKPA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.289984934 (510h12m27.78241738s) [noise=true , address=145.79.227.17:4972 , piece_id=6WR5NCADHHPESCH67KDOV6N53XO7YYUHNNBIIC6GWVH4PJAG7TDQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.290086500 (510h12m27.782518942s) [noise=true , address=145.79.227.15:4873 , piece_id=D65YZCS3VMHP3DHD6WERKCHVC2YYRBSMLKSRA2AJAQVCUPBJQFXQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.290675380 (510h12m27.783107832s) [noise=true , address=77.90.31.13:4628 , piece_id=SXGD6NSGEI52DQQXRZCSKIEIG6UJ6UIJKWJ4KGEMSIPR2WN26IIA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.291221870 (510h12m27.783654312s) [noise=true , address=77.90.31.10:4518 , piece_id=I3G2CDKK3JDE6LGWX2UWAK4HQXK36QK4NNM5MSAZ3WOLLHAAX54Q, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.291949223 (510h12m27.784381664s) [noise=true , address=77.90.31.17:4772 , piece_id=PGH6HN53GMVZVAD5EIIL7FOXQEEO75F2M6Q5CLRRMETTRPCG7SQA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.292804768 (510h12m27.785237213s) [noise=true , address=45.91.116.12:4226 , piece_id=BQJNBSOYHLT62KBHHHIQFNZN2TGQPQLDDQBRV5ZZITCRF5BMMF5Q, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.293727455 (510h12m27.786159902s) [noise=true , address=145.79.227.13:4802 , piece_id=YSQIBFXORIFMTBAX45FPCK3G5QZFXOGMV2PCELJTKOIJA7NIE6AQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.294300159 (510h12m27.786732604s) [noise=true , address=77.90.31.21:4878 , piece_id=COSV4GEXIHOKT6IGQSI6Q5BIZON6MKRCOOAJGXDCVCNXUD5PNSMQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.929478128 (510h12m28.421910557s) [upload=448816]: Next returned nil segment Sep 19 20:20:07.929507048 (510h12m28.421939499s) [upload=448816]: Waiting for error group managing segments... Sep 19 20:20:08.068760885 (510h12m28.561193322s) [noise=true , address=145.79.227.4:4187 , piece_id=5DH3FZIS7T26ZYWCTF3HODHTYCEWMDF6CNZUA6MVSLJJIWIGUIGA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.068976518 (510h12m28.561409044s) [noise=true , address=145.79.227.13:4801 , piece_id=WWBMH6SBFZWJHGLBYK6Q4LHV6VACKZTPH4LQELZPVXKORUQJ56IA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.073031709 (510h12m28.565464144s) [noise=true , address=145.79.227.14:4838 , piece_id=XELCYITGH37HWVBO5AHX6CFTWI5WBX2EGD6GYTBHTL4VZGDVDOJA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.074730788 (510h12m28.567163241s) [noise=true , address=145.79.227.5:4222 , piece_id=MSAWY6N63IL5FHT5GCYAHH7HVNW7B2FJB4U5TFHY2PU7WFKRCKBQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.079043049 (510h12m28.571475501s) [noise=true , address=145.79.227.13:4802 , piece_id=YSQIBFXORIFMTBAX45FPCK3G5QZFXOGMV2PCELJTKOIJA7NIE6AQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081311679 (510h12m28.573744118s) [noise=true , address=145.79.227.16:4912 , piece_id=75ZJUOXV7CF5U6O5ZQ5RFI6NVWEPY7DW3DETBSRXHFBYDU3RHKPA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081720773 (510h12m28.574153202s) [noise=true , address=145.79.227.15:4873 , piece_id=D65YZCS3VMHP3DHD6WERKCHVC2YYRBSMLKSRA2AJAQVCUPBJQFXQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081791531 (510h12m28.574223963s) [noise=true , address=145.79.227.7:4293 , piece_id=W7FHTBVEIEMWSVWGOFDD4DQQDRHXFEIQKLG35HBEB5WX3MHVODWA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.085736522 (510h12m28.57816896s) [noise=true , address=145.79.227.17:4972 , piece_id=6WR5NCADHHPESCH67KDOV6N53XO7YYUHNNBIIC6GWVH4PJAG7TDQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:36.855275377 (510h12m57.347707828s) [noise=true , address=77.90.31.21:4878 , piece_id=COSV4GEXIHOKT6IGQSI6Q5BIZON6MKRCOOAJGXDCVCNXUD5PNSMQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection; write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection Sep 19 20:20:36.855369368 (510h12m57.347801811s) [noise=true , address=77.90.31.13:4628 , piece_id=SXGD6NSGEI52DQQXRZCSKIEIG6UJ6UIJKWJ4KGEMSIPR2WN26IIA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection; write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection Sep 19 20:20:36.855469269 (510h12m57.347901698s) [noise=true , address=77.90.31.19:4842 , piece_id=OLGYQUMO4Z45GY7LWWMJZEZRVROHYVRH37VYRKTRHBNMBAIFNRJA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection Sep 19 20:20:36.855520514 (510h12m57.347952945s) [noise=true , address=45.91.116.4:4939 , piece_id=APAW4FND2AXUZB3KTMLR64RM5PTFJ3ELCAJKOHLYWLBTZTZYT5FQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection; write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection Sep 19 20:20:36.855531580 (510h12m57.347964009s) [noise=true , address=77.90.31.10:4518 , piece_id=I3G2CDKK3JDE6LGWX2UWAK4HQXK36QK4NNM5MSAZ3WOLLHAAX54Q, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection; write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection Sep 19 20:20:36.855594871 (510h12m57.348027299s) [noise=true , address=45.91.116.12:4226 , piece_id=BQJNBSOYHLT62KBHHHIQFNZN2TGQPQLDDQBRV5ZZITCRF5BMMF5Q, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: context deadline exceeded; context deadline exceeded Sep 19 20:20:36.855602167 (510h12m57.348034595s) [noise=true , address=77.90.31.17:4772 , piece_id=PGH6HN53GMVZVAD5EIIL7FOXQEEO75F2M6Q5CLRRMETTRPCG7SQA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 139.59.144.22:36208->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:20:36.855620642 (510h12m57.348053072s) [seg_pos={0 0}, upload=448816]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 19 20:20:36.969452448 (510h12m57.461884952s) [upload=448816]: Done uploading segments... Sep 19 20:20:36.969504159 (510h12m57.461936608s) [upload=448816]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.436303387s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............................................XXXXXXXXX....XX.] duration: 3.059619094s FAIL (2025-09-19 20:21:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.059645285s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..........................................X..XXXXXXXXX.XX.X.] duration: 2.566678013s FAIL (2025-09-19 20:21:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== m27.772288119s) [seg_pos={0 0}, upload=448816]: Begin upload segment... Sep 19 20:20:07.279923237 (510h12m27.772355692s) [seg_pos={0 0}, upload=448816]: Done begin upload segment. Sep 19 20:20:07.279990206 (510h12m27.772422659s) [noise=true , address=45.91.116.4:4939 , piece_id=APAW4FND2AXUZB3KTMLR64RM5PTFJ3ELCAJKOHLYWLBTZTZYT5FQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.280530093 (510h12m27.772962545s) [noise=true , address=145.79.227.4:4187 , piece_id=5DH3FZIS7T26ZYWCTF3HODHTYCEWMDF6CNZUA6MVSLJJIWIGUIGA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.280907172 (510h12m27.773339624s) [noise=true , address=145.79.227.14:4838 , piece_id=XELCYITGH37HWVBO5AHX6CFTWI5WBX2EGD6GYTBHTL4VZGDVDOJA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.281250541 (510h12m27.773682993s) [noise=true , address=145.79.227.13:4801 , piece_id=WWBMH6SBFZWJHGLBYK6Q4LHV6VACKZTPH4LQELZPVXKORUQJ56IA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.281582509 (510h12m27.774014961s) [noise=true , address=145.79.227.5:4222 , piece_id=MSAWY6N63IL5FHT5GCYAHH7HVNW7B2FJB4U5TFHY2PU7WFKRCKBQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282001174 (510h12m27.774433629s) [noise=true , address=77.90.31.19:4842 , piece_id=OLGYQUMO4Z45GY7LWWMJZEZRVROHYVRH37VYRKTRHBNMBAIFNRJA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282350878 (510h12m27.774783333s) [noise=true , address=145.79.227.7:4293 , piece_id=W7FHTBVEIEMWSVWGOFDD4DQQDRHXFEIQKLG35HBEB5WX3MHVODWA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282718781 (510h12m27.775151235s) [noise=true , address=145.79.227.16:4912 , piece_id=75ZJUOXV7CF5U6O5ZQ5RFI6NVWEPY7DW3DETBSRXHFBYDU3RHKPA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.289984934 (510h12m27.78241738s) [noise=true , address=145.79.227.17:4972 , piece_id=6WR5NCADHHPESCH67KDOV6N53XO7YYUHNNBIIC6GWVH4PJAG7TDQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.290086500 (510h12m27.782518942s) [noise=true , address=145.79.227.15:4873 , piece_id=D65YZCS3VMHP3DHD6WERKCHVC2YYRBSMLKSRA2AJAQVCUPBJQFXQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.290675380 (510h12m27.783107832s) [noise=true , address=77.90.31.13:4628 , piece_id=SXGD6NSGEI52DQQXRZCSKIEIG6UJ6UIJKWJ4KGEMSIPR2WN26IIA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.291221870 (510h12m27.783654312s) [noise=true , address=77.90.31.10:4518 , piece_id=I3G2CDKK3JDE6LGWX2UWAK4HQXK36QK4NNM5MSAZ3WOLLHAAX54Q, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.291949223 (510h12m27.784381664s) [noise=true , address=77.90.31.17:4772 , piece_id=PGH6HN53GMVZVAD5EIIL7FOXQEEO75F2M6Q5CLRRMETTRPCG7SQA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.292804768 (510h12m27.785237213s) [noise=true , address=45.91.116.12:4226 , piece_id=BQJNBSOYHLT62KBHHHIQFNZN2TGQPQLDDQBRV5ZZITCRF5BMMF5Q, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.293727455 (510h12m27.786159902s) [noise=true , address=145.79.227.13:4802 , piece_id=YSQIBFXORIFMTBAX45FPCK3G5QZFXOGMV2PCELJTKOIJA7NIE6AQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.294300159 (510h12m27.786732604s) [noise=true , address=77.90.31.21:4878 , piece_id=COSV4GEXIHOKT6IGQSI6Q5BIZON6MKRCOOAJGXDCVCNXUD5PNSMQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.929478128 (510h12m28.421910557s) [upload=448816]: Next returned nil segment Sep 19 20:20:07.929507048 (510h12m28.421939499s) [upload=448816]: Waiting for error group managing segments... Sep 19 20:20:08.068760885 (510h12m28.561193322s) [noise=true , address=145.79.227.4:4187 , piece_id=5DH3FZIS7T26ZYWCTF3HODHTYCEWMDF6CNZUA6MVSLJJIWIGUIGA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.068976518 (510h12m28.561409044s) [noise=true , address=145.79.227.13:4801 , piece_id=WWBMH6SBFZWJHGLBYK6Q4LHV6VACKZTPH4LQELZPVXKORUQJ56IA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.073031709 (510h12m28.565464144s) [noise=true , address=145.79.227.14:4838 , piece_id=XELCYITGH37HWVBO5AHX6CFTWI5WBX2EGD6GYTBHTL4VZGDVDOJA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.074730788 (510h12m28.567163241s) [noise=true , address=145.79.227.5:4222 , piece_id=MSAWY6N63IL5FHT5GCYAHH7HVNW7B2FJB4U5TFHY2PU7WFKRCKBQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.079043049 (510h12m28.571475501s) [noise=true , address=145.79.227.13:4802 , piece_id=YSQIBFXORIFMTBAX45FPCK3G5QZFXOGMV2PCELJTKOIJA7NIE6AQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081311679 (510h12m28.573744118s) [noise=true , address=145.79.227.16:4912 , piece_id=75ZJUOXV7CF5U6O5ZQ5RFI6NVWEPY7DW3DETBSRXHFBYDU3RHKPA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081720773 (510h12m28.574153202s) [noise=true , address=145.79.227.15:4873 , piece_id=D65YZCS3VMHP3DHD6WERKCHVC2YYRBSMLKSRA2AJAQVCUPBJQFXQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081791531 (510h12m28.574223963s) [noise=true , address=145.79.227.7:4293 , piece_id=W7FHTBVEIEMWSVWGOFDD4DQQDRHXFEIQKLG35HBEB5WX3MHVODWA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.085736522 (510h12m28.57816896s) [noise=true , address=145.79.227.17:4972 , piece_id=6WR5NCADHHPESCH67KDOV6N53XO7YYUHNNBIIC6GWVH4PJAG7TDQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:36.855275377 (510h12m57.347707828s) [noise=true , address=77.90.31.21:4878 , piece_id=COSV4GEXIHOKT6IGQSI6Q5BIZON6MKRCOOAJGXDCVCNXUD5PNSMQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection; write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection Sep 19 20:20:36.855369368 (510h12m57.347801811s) [noise=true , address=77.90.31.13:4628 , piece_id=SXGD6NSGEI52DQQXRZCSKIEIG6UJ6UIJKWJ4KGEMSIPR2WN26IIA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection; write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection Sep 19 20:20:36.855469269 (510h12m57.347901698s) [noise=true , address=77.90.31.19:4842 , piece_id=OLGYQUMO4Z45GY7LWWMJZEZRVROHYVRH37VYRKTRHBNMBAIFNRJA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection Sep 19 20:20:36.855520514 (510h12m57.347952945s) [noise=true , address=45.91.116.4:4939 , piece_id=APAW4FND2AXUZB3KTMLR64RM5PTFJ3ELCAJKOHLYWLBTZTZYT5FQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection; write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection Sep 19 20:20:36.855531580 (510h12m57.347964009s) [noise=true , address=77.90.31.10:4518 , piece_id=I3G2CDKK3JDE6LGWX2UWAK4HQXK36QK4NNM5MSAZ3WOLLHAAX54Q, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection; write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection Sep 19 20:20:36.855594871 (510h12m57.348027299s) [noise=true , address=45.91.116.12:4226 , piece_id=BQJNBSOYHLT62KBHHHIQFNZN2TGQPQLDDQBRV5ZZITCRF5BMMF5Q, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: context deadline exceeded; context deadline exceeded Sep 19 20:20:36.855602167 (510h12m57.348034595s) [noise=true , address=77.90.31.17:4772 , piece_id=PGH6HN53GMVZVAD5EIIL7FOXQEEO75F2M6Q5CLRRMETTRPCG7SQA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 139.59.144.22:36208->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:20:36.855620642 (510h12m57.348053072s) [seg_pos={0 0}, upload=448816]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 19 20:20:36.969452448 (510h12m57.461884952s) [upload=448816]: Done uploading segments... Sep 19 20:20:36.969504159 (510h12m57.461936608s) [upload=448816]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.566703246s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.............................................XXXXXXXXX....XX] duration: 30.001303287s FAIL (2025-09-19 20:21:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001312965s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..............................................XXXXXXXXX....X] duration: 30.000472811s FAIL (2025-09-19 20:20:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000567555s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...........................................X..XXXXXXXXX.XX.X] duration: 30.116043764s FAIL (2025-09-19 20:20:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== m27.772288119s) [seg_pos={0 0}, upload=448816]: Begin upload segment... Sep 19 20:20:07.279923237 (510h12m27.772355692s) [seg_pos={0 0}, upload=448816]: Done begin upload segment. Sep 19 20:20:07.279990206 (510h12m27.772422659s) [noise=true , address=45.91.116.4:4939 , piece_id=APAW4FND2AXUZB3KTMLR64RM5PTFJ3ELCAJKOHLYWLBTZTZYT5FQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.280530093 (510h12m27.772962545s) [noise=true , address=145.79.227.4:4187 , piece_id=5DH3FZIS7T26ZYWCTF3HODHTYCEWMDF6CNZUA6MVSLJJIWIGUIGA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.280907172 (510h12m27.773339624s) [noise=true , address=145.79.227.14:4838 , piece_id=XELCYITGH37HWVBO5AHX6CFTWI5WBX2EGD6GYTBHTL4VZGDVDOJA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.281250541 (510h12m27.773682993s) [noise=true , address=145.79.227.13:4801 , piece_id=WWBMH6SBFZWJHGLBYK6Q4LHV6VACKZTPH4LQELZPVXKORUQJ56IA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.281582509 (510h12m27.774014961s) [noise=true , address=145.79.227.5:4222 , piece_id=MSAWY6N63IL5FHT5GCYAHH7HVNW7B2FJB4U5TFHY2PU7WFKRCKBQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282001174 (510h12m27.774433629s) [noise=true , address=77.90.31.19:4842 , piece_id=OLGYQUMO4Z45GY7LWWMJZEZRVROHYVRH37VYRKTRHBNMBAIFNRJA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282350878 (510h12m27.774783333s) [noise=true , address=145.79.227.7:4293 , piece_id=W7FHTBVEIEMWSVWGOFDD4DQQDRHXFEIQKLG35HBEB5WX3MHVODWA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.282718781 (510h12m27.775151235s) [noise=true , address=145.79.227.16:4912 , piece_id=75ZJUOXV7CF5U6O5ZQ5RFI6NVWEPY7DW3DETBSRXHFBYDU3RHKPA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.289984934 (510h12m27.78241738s) [noise=true , address=145.79.227.17:4972 , piece_id=6WR5NCADHHPESCH67KDOV6N53XO7YYUHNNBIIC6GWVH4PJAG7TDQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.290086500 (510h12m27.782518942s) [noise=true , address=145.79.227.15:4873 , piece_id=D65YZCS3VMHP3DHD6WERKCHVC2YYRBSMLKSRA2AJAQVCUPBJQFXQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.290675380 (510h12m27.783107832s) [noise=true , address=77.90.31.13:4628 , piece_id=SXGD6NSGEI52DQQXRZCSKIEIG6UJ6UIJKWJ4KGEMSIPR2WN26IIA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.291221870 (510h12m27.783654312s) [noise=true , address=77.90.31.10:4518 , piece_id=I3G2CDKK3JDE6LGWX2UWAK4HQXK36QK4NNM5MSAZ3WOLLHAAX54Q, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.291949223 (510h12m27.784381664s) [noise=true , address=77.90.31.17:4772 , piece_id=PGH6HN53GMVZVAD5EIIL7FOXQEEO75F2M6Q5CLRRMETTRPCG7SQA, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.292804768 (510h12m27.785237213s) [noise=true , address=45.91.116.12:4226 , piece_id=BQJNBSOYHLT62KBHHHIQFNZN2TGQPQLDDQBRV5ZZITCRF5BMMF5Q, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.293727455 (510h12m27.786159902s) [noise=true , address=145.79.227.13:4802 , piece_id=YSQIBFXORIFMTBAX45FPCK3G5QZFXOGMV2PCELJTKOIJA7NIE6AQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.294300159 (510h12m27.786732604s) [noise=true , address=77.90.31.21:4878 , piece_id=COSV4GEXIHOKT6IGQSI6Q5BIZON6MKRCOOAJGXDCVCNXUD5PNSMQ, seg_pos={0 0}, upload=448816]: Uploading piece... Sep 19 20:20:07.929478128 (510h12m28.421910557s) [upload=448816]: Next returned nil segment Sep 19 20:20:07.929507048 (510h12m28.421939499s) [upload=448816]: Waiting for error group managing segments... Sep 19 20:20:08.068760885 (510h12m28.561193322s) [noise=true , address=145.79.227.4:4187 , piece_id=5DH3FZIS7T26ZYWCTF3HODHTYCEWMDF6CNZUA6MVSLJJIWIGUIGA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.068976518 (510h12m28.561409044s) [noise=true , address=145.79.227.13:4801 , piece_id=WWBMH6SBFZWJHGLBYK6Q4LHV6VACKZTPH4LQELZPVXKORUQJ56IA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.073031709 (510h12m28.565464144s) [noise=true , address=145.79.227.14:4838 , piece_id=XELCYITGH37HWVBO5AHX6CFTWI5WBX2EGD6GYTBHTL4VZGDVDOJA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.074730788 (510h12m28.567163241s) [noise=true , address=145.79.227.5:4222 , piece_id=MSAWY6N63IL5FHT5GCYAHH7HVNW7B2FJB4U5TFHY2PU7WFKRCKBQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.079043049 (510h12m28.571475501s) [noise=true , address=145.79.227.13:4802 , piece_id=YSQIBFXORIFMTBAX45FPCK3G5QZFXOGMV2PCELJTKOIJA7NIE6AQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081311679 (510h12m28.573744118s) [noise=true , address=145.79.227.16:4912 , piece_id=75ZJUOXV7CF5U6O5ZQ5RFI6NVWEPY7DW3DETBSRXHFBYDU3RHKPA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081720773 (510h12m28.574153202s) [noise=true , address=145.79.227.15:4873 , piece_id=D65YZCS3VMHP3DHD6WERKCHVC2YYRBSMLKSRA2AJAQVCUPBJQFXQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.081791531 (510h12m28.574223963s) [noise=true , address=145.79.227.7:4293 , piece_id=W7FHTBVEIEMWSVWGOFDD4DQQDRHXFEIQKLG35HBEB5WX3MHVODWA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:08.085736522 (510h12m28.57816896s) [noise=true , address=145.79.227.17:4972 , piece_id=6WR5NCADHHPESCH67KDOV6N53XO7YYUHNNBIIC6GWVH4PJAG7TDQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: <nil> Sep 19 20:20:36.855275377 (510h12m57.347707828s) [noise=true , address=77.90.31.21:4878 , piece_id=COSV4GEXIHOKT6IGQSI6Q5BIZON6MKRCOOAJGXDCVCNXUD5PNSMQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection; write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53104->77.90.31.21:4878: use of closed network connection Sep 19 20:20:36.855369368 (510h12m57.347801811s) [noise=true , address=77.90.31.13:4628 , piece_id=SXGD6NSGEI52DQQXRZCSKIEIG6UJ6UIJKWJ4KGEMSIPR2WN26IIA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection; write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40178->77.90.31.13:4628: use of closed network connection Sep 19 20:20:36.855469269 (510h12m57.347901698s) [noise=true , address=77.90.31.19:4842 , piece_id=OLGYQUMO4Z45GY7LWWMJZEZRVROHYVRH37VYRKTRHBNMBAIFNRJA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48810->77.90.31.19:4842: use of closed network connection Sep 19 20:20:36.855520514 (510h12m57.347952945s) [noise=true , address=45.91.116.4:4939 , piece_id=APAW4FND2AXUZB3KTMLR64RM5PTFJ3ELCAJKOHLYWLBTZTZYT5FQ, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection; write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52132->45.91.116.4:4939: use of closed network connection Sep 19 20:20:36.855531580 (510h12m57.347964009s) [noise=true , address=77.90.31.10:4518 , piece_id=I3G2CDKK3JDE6LGWX2UWAK4HQXK36QK4NNM5MSAZ3WOLLHAAX54Q, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection; write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49702->77.90.31.10:4518: use of closed network connection Sep 19 20:20:36.855594871 (510h12m57.348027299s) [noise=true , address=45.91.116.12:4226 , piece_id=BQJNBSOYHLT62KBHHHIQFNZN2TGQPQLDDQBRV5ZZITCRF5BMMF5Q, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: context deadline exceeded; context deadline exceeded Sep 19 20:20:36.855602167 (510h12m57.348034595s) [noise=true , address=77.90.31.17:4772 , piece_id=PGH6HN53GMVZVAD5EIIL7FOXQEEO75F2M6Q5CLRRMETTRPCG7SQA, seg_pos={0 0}, upload=448816]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 139.59.144.22:36208->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:20:36.855620642 (510h12m57.348053072s) [seg_pos={0 0}, upload=448816]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 19 20:20:36.969452448 (510h12m57.461884952s) [upload=448816]: Done uploading segments... Sep 19 20:20:36.969504159 (510h12m57.461936608s) [upload=448816]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.116076729s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...............................................XXXXXXXXX....] duration: 3.076082278s FAIL (2025-09-19 20:19:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.076144173s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [............................................X..XXXXXXXXX.XX.] duration: 2.538136682s FAIL (2025-09-19 20:18:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 19 20:17:52.276654748 (510h10m12.769087202s) [upload=448785]: Starting upload Sep 19 20:17:52.276741423 (510h10m12.76917387s) [upload=448785]: Done starting upload Sep 19 20:17:52.277019250 (510h10m12.769451679s) [upload=448785]: Uploading segments... Sep 19 20:17:52.277332255 (510h10m12.769764704s) [upload=448785]: Got next segment. Inline: false Sep 19 20:17:52.277408044 (510h10m12.769840505s) [upload=448785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:17:52.277414350 (510h10m12.769846799s) [upload=448785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:17:52.436551244 (510h10m12.928983676s) [seg_pos={0 0}, upload=448785]: Begin upload segment... Sep 19 20:17:52.436607694 (510h10m12.929040144s) [seg_pos={0 0}, upload=448785]: Done begin upload segment. Sep 19 20:17:52.436647854 (510h10m12.929080307s) [noise=true , address=45.91.116.11:4119 , piece_id=2WTNN5O3GVBTU7GAW6UMNOSW3Y4O3MCFTWHAY2PWAOAZQAFAUPCQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.440372844 (510h10m12.932805296s) [noise=true , address=77.90.31.15:4699 , piece_id=DEOLYLTE3V5WYNIKDODXGSBX7LHJ3MTMN24KPVALKN67O73SQ3PA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.440876653 (510h10m12.933309095s) [noise=true , address=77.90.31.14:4662 , piece_id=3FFSUUZA6VABRFO7ICHFQE74CM6PMEL3PRHQM5UGFZCKS5KTCJMQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.441234663 (510h10m12.933667106s) [noise=true , address=45.91.116.14:4191 , piece_id=MDXNJKT7IFW643WPQWNOA4TFAWJE5SW3OTKUHYZ6NDLR4LQ26FLQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.441541910 (510h10m12.933974354s) [noise=true , address=77.90.31.19:4842 , piece_id=MIBS3MWU2OJM33UQHMPGMTKO4DFFBJMSHVH5PZGINWBAYVYQN3TQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.441887141 (510h10m12.934319586s) [noise=true , address=145.79.227.4:4186 , piece_id=E64LQJ7XCT4FOPMQUVM4HKCU7THQWGFUAIH7HJRPZN7DCJTGYNRQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.442272217 (510h10m12.934704664s) [noise=true , address=45.91.116.9:4261 , piece_id=WTXLEEHNO4NCN7VGP5MVWAPABQIBIBOHMBJYDQZJ6SHPN3H26NDA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.442558728 (510h10m12.934991174s) [noise=true , address=45.91.116.6:4974 , piece_id=FYBKMT7CSHDCV6UEP6REKZ2BGBBOMXPUCBXG4J7255OT7KL4XWEA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.442807824 (510h10m12.935240265s) [noise=true , address=145.79.227.12:4767 , piece_id=F4IHUH4EJ2D2AM3GRDZGCDRDUF6PMSYMMRPEI3EHXKATN3Y6KJUA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.443134734 (510h10m12.935567176s) [noise=true , address=77.90.31.13:4626 , piece_id=AEXUF4I56DWLTJ662UO2QR7DQ2GIQCNJDDFEVAFRGDARUP6OCOPA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.443507872 (510h10m12.935940318s) [noise=true , address=77.90.31.21:4879 , piece_id=SGNZCRZSFZWSAEL27AMWH4XOV6ATWSDFGK4TURYUJ62P2E2R2G5Q, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.443853310 (510h10m12.936285755s) [noise=true , address=77.90.31.11:4553 , piece_id=REEMFNAVWZ67CR5IWQZVPYBZS3EOHVAGCYYG6QZ7CKELOE2HWSXQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.444300956 (510h10m12.936733398s) [noise=true , address=77.90.31.16:4735 , piece_id=RASG76FXP7IQZSQUJH57J2W3AKOXO5MIHBGKQZIBA54WUNUXIR7Q, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.444769173 (510h10m12.937201614s) [noise=true , address=45.91.116.4:4940 , piece_id=RFOQHL7QZCEGYTRQQPI3LCB4RZJT5KFDSBUVDZCEEQHPASCF4RVA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.445077587 (510h10m12.937510032s) [noise=true , address=145.79.227.11:4730 , piece_id=7COT6FLTRIGLCSLE2BK7NR5YTDRAFCFD7QFQDIH4A5JSKJ57X5OA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.445339049 (510h10m12.93777149s) [noise=true , address=145.79.227.3:4150 , piece_id=OYPFQVUMJQMUX3YRRSEXX7TCX3RSPZBBNKCDBA6PTOTG6OXK2TDA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:53.106175293 (510h10m13.598607745s) [upload=448785]: Next returned nil segment Sep 19 20:17:53.106198527 (510h10m13.598630977s) [upload=448785]: Waiting for error group managing segments... Sep 19 20:17:53.244889288 (510h10m13.73732172s) [noise=true , address=145.79.227.12:4767 , piece_id=F4IHUH4EJ2D2AM3GRDZGCDRDUF6PMSYMMRPEI3EHXKATN3Y6KJUA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:17:53.256721765 (510h10m13.749154211s) [noise=true , address=145.79.227.4:4186 , piece_id=E64LQJ7XCT4FOPMQUVM4HKCU7THQWGFUAIH7HJRPZN7DCJTGYNRQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:17:53.276951272 (510h10m13.769383716s) [noise=true , address=145.79.227.11:4730 , piece_id=7COT6FLTRIGLCSLE2BK7NR5YTDRAFCFD7QFQDIH4A5JSKJ57X5OA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:17:53.294905967 (510h10m13.78733841s) [noise=true , address=145.79.227.3:4150 , piece_id=OYPFQVUMJQMUX3YRRSEXX7TCX3RSPZBBNKCDBA6PTOTG6OXK2TDA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:14.703132386 (510h10m35.195564836s) [noise=true , address=77.90.31.13:4626 , piece_id=AEXUF4I56DWLTJ662UO2QR7DQ2GIQCNJDDFEVAFRGDARUP6OCOPA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:14.924270957 (510h10m35.4167034s) [noise=true , address=45.91.116.11:4119 , piece_id=2WTNN5O3GVBTU7GAW6UMNOSW3Y4O3MCFTWHAY2PWAOAZQAFAUPCQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:16.243571138 (510h10m36.736003574s) [noise=true , address=77.90.31.11:4553 , piece_id=REEMFNAVWZ67CR5IWQZVPYBZS3EOHVAGCYYG6QZ7CKELOE2HWSXQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:16.457232596 (510h10m36.949665029s) [noise=true , address=77.90.31.19:4842 , piece_id=MIBS3MWU2OJM33UQHMPGMTKO4DFFBJMSHVH5PZGINWBAYVYQN3TQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:17.757420421 (510h10m38.249852866s) [noise=true , address=77.90.31.16:4735 , piece_id=RASG76FXP7IQZSQUJH57J2W3AKOXO5MIHBGKQZIBA54WUNUXIR7Q, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:20.421389039 (510h10m40.91382148s) [noise=true , address=45.91.116.9:4261 , piece_id=WTXLEEHNO4NCN7VGP5MVWAPABQIBIBOHMBJYDQZJ6SHPN3H26NDA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:21.966031203 (510h10m42.458463635s) [noise=true , address=45.91.116.14:4191 , piece_id=MDXNJKT7IFW643WPQWNOA4TFAWJE5SW3OTKUHYZ6NDLR4LQ26FLQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966182281 (510h10m42.458614723s) [noise=true , address=77.90.31.21:4879 , piece_id=SGNZCRZSFZWSAEL27AMWH4XOV6ATWSDFGK4TURYUJ62P2E2R2G5Q, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 139.59.144.22:54700->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966230153 (510h10m42.458662595s) [noise=true , address=77.90.31.14:4662 , piece_id=3FFSUUZA6VABRFO7ICHFQE74CM6PMEL3PRHQM5UGFZCKS5KTCJMQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966352542 (510h10m42.458784976s) [noise=true , address=77.90.31.15:4699 , piece_id=DEOLYLTE3V5WYNIKDODXGSBX7LHJ3MTMN24KPVALKN67O73SQ3PA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:44466->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966382731 (510h10m42.458815166s) [noise=true , address=45.91.116.4:4940 , piece_id=RFOQHL7QZCEGYTRQQPI3LCB4RZJT5KFDSBUVDZCEEQHPASCF4RVA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: read tcp 139.59.144.22:44440->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966391144 (510h10m42.458823586s) [noise=true , address=45.91.116.6:4974 , piece_id=FYBKMT7CSHDCV6UEP6REKZ2BGBBOMXPUCBXG4J7255OT7KL4XWEA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: read tcp 139.59.144.22:36356->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966420184 (510h10m42.458852633s) [seg_pos={0 0}, upload=448785]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 19 20:18:22.081476703 (510h10m42.573909139s) [upload=448785]: Done uploading segments... Sep 19 20:18:22.081509202 (510h10m42.573941645s) [upload=448785]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.53817044s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [................................................XXXXXXXXX...] duration: 2.964697819s FAIL (2025-09-19 20:18:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.964725787s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.............................................X..XXXXXXXXX.XX] duration: 30.117230655s FAIL (2025-09-19 20:18:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 19 20:17:52.276654748 (510h10m12.769087202s) [upload=448785]: Starting upload Sep 19 20:17:52.276741423 (510h10m12.76917387s) [upload=448785]: Done starting upload Sep 19 20:17:52.277019250 (510h10m12.769451679s) [upload=448785]: Uploading segments... Sep 19 20:17:52.277332255 (510h10m12.769764704s) [upload=448785]: Got next segment. Inline: false Sep 19 20:17:52.277408044 (510h10m12.769840505s) [upload=448785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:17:52.277414350 (510h10m12.769846799s) [upload=448785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:17:52.436551244 (510h10m12.928983676s) [seg_pos={0 0}, upload=448785]: Begin upload segment... Sep 19 20:17:52.436607694 (510h10m12.929040144s) [seg_pos={0 0}, upload=448785]: Done begin upload segment. Sep 19 20:17:52.436647854 (510h10m12.929080307s) [noise=true , address=45.91.116.11:4119 , piece_id=2WTNN5O3GVBTU7GAW6UMNOSW3Y4O3MCFTWHAY2PWAOAZQAFAUPCQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.440372844 (510h10m12.932805296s) [noise=true , address=77.90.31.15:4699 , piece_id=DEOLYLTE3V5WYNIKDODXGSBX7LHJ3MTMN24KPVALKN67O73SQ3PA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.440876653 (510h10m12.933309095s) [noise=true , address=77.90.31.14:4662 , piece_id=3FFSUUZA6VABRFO7ICHFQE74CM6PMEL3PRHQM5UGFZCKS5KTCJMQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.441234663 (510h10m12.933667106s) [noise=true , address=45.91.116.14:4191 , piece_id=MDXNJKT7IFW643WPQWNOA4TFAWJE5SW3OTKUHYZ6NDLR4LQ26FLQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.441541910 (510h10m12.933974354s) [noise=true , address=77.90.31.19:4842 , piece_id=MIBS3MWU2OJM33UQHMPGMTKO4DFFBJMSHVH5PZGINWBAYVYQN3TQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.441887141 (510h10m12.934319586s) [noise=true , address=145.79.227.4:4186 , piece_id=E64LQJ7XCT4FOPMQUVM4HKCU7THQWGFUAIH7HJRPZN7DCJTGYNRQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.442272217 (510h10m12.934704664s) [noise=true , address=45.91.116.9:4261 , piece_id=WTXLEEHNO4NCN7VGP5MVWAPABQIBIBOHMBJYDQZJ6SHPN3H26NDA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.442558728 (510h10m12.934991174s) [noise=true , address=45.91.116.6:4974 , piece_id=FYBKMT7CSHDCV6UEP6REKZ2BGBBOMXPUCBXG4J7255OT7KL4XWEA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.442807824 (510h10m12.935240265s) [noise=true , address=145.79.227.12:4767 , piece_id=F4IHUH4EJ2D2AM3GRDZGCDRDUF6PMSYMMRPEI3EHXKATN3Y6KJUA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.443134734 (510h10m12.935567176s) [noise=true , address=77.90.31.13:4626 , piece_id=AEXUF4I56DWLTJ662UO2QR7DQ2GIQCNJDDFEVAFRGDARUP6OCOPA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.443507872 (510h10m12.935940318s) [noise=true , address=77.90.31.21:4879 , piece_id=SGNZCRZSFZWSAEL27AMWH4XOV6ATWSDFGK4TURYUJ62P2E2R2G5Q, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.443853310 (510h10m12.936285755s) [noise=true , address=77.90.31.11:4553 , piece_id=REEMFNAVWZ67CR5IWQZVPYBZS3EOHVAGCYYG6QZ7CKELOE2HWSXQ, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.444300956 (510h10m12.936733398s) [noise=true , address=77.90.31.16:4735 , piece_id=RASG76FXP7IQZSQUJH57J2W3AKOXO5MIHBGKQZIBA54WUNUXIR7Q, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.444769173 (510h10m12.937201614s) [noise=true , address=45.91.116.4:4940 , piece_id=RFOQHL7QZCEGYTRQQPI3LCB4RZJT5KFDSBUVDZCEEQHPASCF4RVA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.445077587 (510h10m12.937510032s) [noise=true , address=145.79.227.11:4730 , piece_id=7COT6FLTRIGLCSLE2BK7NR5YTDRAFCFD7QFQDIH4A5JSKJ57X5OA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:52.445339049 (510h10m12.93777149s) [noise=true , address=145.79.227.3:4150 , piece_id=OYPFQVUMJQMUX3YRRSEXX7TCX3RSPZBBNKCDBA6PTOTG6OXK2TDA, seg_pos={0 0}, upload=448785]: Uploading piece... Sep 19 20:17:53.106175293 (510h10m13.598607745s) [upload=448785]: Next returned nil segment Sep 19 20:17:53.106198527 (510h10m13.598630977s) [upload=448785]: Waiting for error group managing segments... Sep 19 20:17:53.244889288 (510h10m13.73732172s) [noise=true , address=145.79.227.12:4767 , piece_id=F4IHUH4EJ2D2AM3GRDZGCDRDUF6PMSYMMRPEI3EHXKATN3Y6KJUA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:17:53.256721765 (510h10m13.749154211s) [noise=true , address=145.79.227.4:4186 , piece_id=E64LQJ7XCT4FOPMQUVM4HKCU7THQWGFUAIH7HJRPZN7DCJTGYNRQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:17:53.276951272 (510h10m13.769383716s) [noise=true , address=145.79.227.11:4730 , piece_id=7COT6FLTRIGLCSLE2BK7NR5YTDRAFCFD7QFQDIH4A5JSKJ57X5OA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:17:53.294905967 (510h10m13.78733841s) [noise=true , address=145.79.227.3:4150 , piece_id=OYPFQVUMJQMUX3YRRSEXX7TCX3RSPZBBNKCDBA6PTOTG6OXK2TDA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:14.703132386 (510h10m35.195564836s) [noise=true , address=77.90.31.13:4626 , piece_id=AEXUF4I56DWLTJ662UO2QR7DQ2GIQCNJDDFEVAFRGDARUP6OCOPA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:14.924270957 (510h10m35.4167034s) [noise=true , address=45.91.116.11:4119 , piece_id=2WTNN5O3GVBTU7GAW6UMNOSW3Y4O3MCFTWHAY2PWAOAZQAFAUPCQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:16.243571138 (510h10m36.736003574s) [noise=true , address=77.90.31.11:4553 , piece_id=REEMFNAVWZ67CR5IWQZVPYBZS3EOHVAGCYYG6QZ7CKELOE2HWSXQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:16.457232596 (510h10m36.949665029s) [noise=true , address=77.90.31.19:4842 , piece_id=MIBS3MWU2OJM33UQHMPGMTKO4DFFBJMSHVH5PZGINWBAYVYQN3TQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:17.757420421 (510h10m38.249852866s) [noise=true , address=77.90.31.16:4735 , piece_id=RASG76FXP7IQZSQUJH57J2W3AKOXO5MIHBGKQZIBA54WUNUXIR7Q, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:20.421389039 (510h10m40.91382148s) [noise=true , address=45.91.116.9:4261 , piece_id=WTXLEEHNO4NCN7VGP5MVWAPABQIBIBOHMBJYDQZJ6SHPN3H26NDA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: <nil> Sep 19 20:18:21.966031203 (510h10m42.458463635s) [noise=true , address=45.91.116.14:4191 , piece_id=MDXNJKT7IFW643WPQWNOA4TFAWJE5SW3OTKUHYZ6NDLR4LQ26FLQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966182281 (510h10m42.458614723s) [noise=true , address=77.90.31.21:4879 , piece_id=SGNZCRZSFZWSAEL27AMWH4XOV6ATWSDFGK4TURYUJ62P2E2R2G5Q, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 139.59.144.22:54700->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966230153 (510h10m42.458662595s) [noise=true , address=77.90.31.14:4662 , piece_id=3FFSUUZA6VABRFO7ICHFQE74CM6PMEL3PRHQM5UGFZCKS5KTCJMQ, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966352542 (510h10m42.458784976s) [noise=true , address=77.90.31.15:4699 , piece_id=DEOLYLTE3V5WYNIKDODXGSBX7LHJ3MTMN24KPVALKN67O73SQ3PA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:44466->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966382731 (510h10m42.458815166s) [noise=true , address=45.91.116.4:4940 , piece_id=RFOQHL7QZCEGYTRQQPI3LCB4RZJT5KFDSBUVDZCEEQHPASCF4RVA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: read tcp 139.59.144.22:44440->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966391144 (510h10m42.458823586s) [noise=true , address=45.91.116.6:4974 , piece_id=FYBKMT7CSHDCV6UEP6REKZ2BGBBOMXPUCBXG4J7255OT7KL4XWEA, seg_pos={0 0}, upload=448785]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: read tcp 139.59.144.22:36356->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:18:21.966420184 (510h10m42.458852633s) [seg_pos={0 0}, upload=448785]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 19 20:18:22.081476703 (510h10m42.573909139s) [upload=448785]: Done uploading segments... Sep 19 20:18:22.081509202 (510h10m42.573941645s) [upload=448785]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.117326845s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................................................XXXXXXXXX..] duration: 18.133406558s FAIL (2025-09-19 20:18:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.133780306s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..............................................X..XXXXXXXXX.X] duration: 30.110362342s FAIL (2025-09-19 20:17:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 19 20:16:47.881391953 (510h9m8.373824399s) [upload=448769]: Starting upload Sep 19 20:16:47.881463151 (510h9m8.373895596s) [upload=448769]: Done starting upload Sep 19 20:16:47.881736029 (510h9m8.374168457s) [upload=448769]: Uploading segments... Sep 19 20:16:47.881772087 (510h9m8.374204515s) [upload=448769]: Got next segment. Inline: false Sep 19 20:16:47.881803802 (510h9m8.374236231s) [upload=448769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:16:47.881807807 (510h9m8.374240235s) [upload=448769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:16:47.996288785 (510h9m8.488721218s) [seg_pos={0 0}, upload=448769]: Begin upload segment... Sep 19 20:16:47.996373071 (510h9m8.488805515s) [seg_pos={0 0}, upload=448769]: Done begin upload segment. Sep 19 20:16:47.996411163 (510h9m8.488843626s) [noise=true , address=77.90.31.12:4590 , piece_id=S54Z6FALE5PGE5AOHJ5N7S5ZZ7YJMR3S2DPKRM2ULRSO3ZH5CSNQ, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.996432645 (510h9m8.488865098s) [noise=true , address=45.91.116.10:4082 , piece_id=3XXBP56ZB4MPVPK4HWJUBDGCLQIVPC3HDVS3Q32WWYP4L3ERMQ4Q, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.996839349 (510h9m8.489271777s) [noise=true , address=45.91.116.10:4083 , piece_id=KM5OF7FZABFOARNPHNE4LS7TJZ35OLXLIMMUR6YHLT4ITHTU3RKA, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.996861049 (510h9m8.489293498s) [noise=true , address=145.79.227.11:4732 , piece_id=WFNQJ4RKM7ET3LWYFOIVZJCLNNAE3FRYH2LYUZN5OEBSFAOB2LTA, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.997260719 (510h9m8.489693171s) [noise=true , address=77.90.31.19:4841 , piece_id=LVH2Z6VRJV5HE3QZQKXAPFHUDOEXWQT5NTBHXCXFLZVX45MI5AEA, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.997655358 (510h9m8.49008781s) [noise=true , address=45.91.116.6:4974 , piece_id=BC2O2HE52JBIAOR7MNFU3KTSQDFH6FSHJTTB3XYO2LMSGMUB2DMQ, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.997894188 (510h9m8.490326619s) [noise=true , address=45.91.116.12:4227 , piece_id=NMJC7LUC37NBYG3QOE3LS5BAXL3WYWSQ4HNVCLCM5ZPF2ZGGDW5A, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.998113309 (510h9m8.490545762s) [noise=true , address=45.91.116.14:4191 , piece_id=A7FE4HO4RNWLZXDHQAZLIIF22UIOUYOYKC5C7MCDWOKPW5EJY2SA, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.998522135 (510h9m8.490954589s) [noise=true , address=45.91.116.4:4940 , piece_id=25HCVTTGHS2IAFNSNTR3MEVZBMX6BOQVLL7VFL6C5LL44FNFGU3A, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.998631044 (510h9m8.491063493s) [noise=true , address=77.90.31.11:4553 , piece_id=HGUISSOWXVMH2C3YLPKMZA3DDLMFHDV6RP34AEVPNVBFASDS46VA, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.998967215 (510h9m8.491399681s) [noise=true , address=45.91.116.7:4011 , piece_id=33SZJZJMI2KDQFNTYQ46LWT65HFDPSUHLS4FTKW5YTYNV4A52JGQ, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.999329487 (510h9m8.49176193s) [noise=true , address=145.79.227.13:4801 , piece_id=SOGVBLDVJLCKSGHBUFYYGB32QEX4HADV3NCUPDR5PWO7E5WSUD3Q, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:47.999659172 (510h9m8.492091626s) [noise=true , address=77.90.31.14:4663 , piece_id=YYEMNYCHHYYSCV73TKR2BUX5QJDKJNG43XDBACEKKWW2RVND5J5Q, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:48.000002487 (510h9m8.492434974s) [noise=true , address=77.90.31.10:4517 , piece_id=K7OLDFV6YMGHAQL6MQ5Q3MRVHFBZ3D3WHTLUAO2NWG5J5OKBT2LA, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:48.000432933 (510h9m8.492865403s) [noise=true , address=45.91.116.9:4263 , piece_id=ALBRL47XM54N32FGD3WDWXGN4NFTBTSLKAAEVTJEQ5P5PE5EWTTA, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:48.000661876 (510h9m8.493094326s) [noise=true , address=145.79.227.17:4971 , piece_id=ZZ6JCFIFOLFSNLTOWBLNXGJZSVGTCVLRERB5FHY3EQMULY4I2UJA, seg_pos={0 0}, upload=448769]: Uploading piece... Sep 19 20:16:48.631754902 (510h9m9.124187358s) [upload=448769]: Next returned nil segment Sep 19 20:16:48.631780659 (510h9m9.124213111s) [upload=448769]: Waiting for error group managing segments... Sep 19 20:16:48.773013983 (510h9m9.265446412s) [noise=true , address=145.79.227.13:4801 , piece_id=SOGVBLDVJLCKSGHBUFYYGB32QEX4HADV3NCUPDR5PWO7E5WSUD3Q, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:16:48.775743226 (510h9m9.268175655s) [noise=true , address=145.79.227.11:4732 , piece_id=WFNQJ4RKM7ET3LWYFOIVZJCLNNAE3FRYH2LYUZN5OEBSFAOB2LTA, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:16:48.780844276 (510h9m9.273276726s) [noise=true , address=145.79.227.17:4971 , piece_id=ZZ6JCFIFOLFSNLTOWBLNXGJZSVGTCVLRERB5FHY3EQMULY4I2UJA, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:08.436999174 (510h9m28.929431606s) [noise=true , address=45.91.116.9:4263 , piece_id=ALBRL47XM54N32FGD3WDWXGN4NFTBTSLKAAEVTJEQ5P5PE5EWTTA, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:09.029125815 (510h9m29.521558244s) [noise=true , address=45.91.116.12:4227 , piece_id=NMJC7LUC37NBYG3QOE3LS5BAXL3WYWSQ4HNVCLCM5ZPF2ZGGDW5A, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:09.154856831 (510h9m29.647289267s) [noise=true , address=45.91.116.10:4083 , piece_id=KM5OF7FZABFOARNPHNE4LS7TJZ35OLXLIMMUR6YHLT4ITHTU3RKA, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:09.379867276 (510h9m29.872299708s) [noise=true , address=77.90.31.12:4590 , piece_id=S54Z6FALE5PGE5AOHJ5N7S5ZZ7YJMR3S2DPKRM2ULRSO3ZH5CSNQ, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:09.399931253 (510h9m29.892363693s) [noise=true , address=45.91.116.4:4940 , piece_id=25HCVTTGHS2IAFNSNTR3MEVZBMX6BOQVLL7VFL6C5LL44FNFGU3A, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:09.726286513 (510h9m30.218718959s) [noise=true , address=45.91.116.6:4974 , piece_id=BC2O2HE52JBIAOR7MNFU3KTSQDFH6FSHJTTB3XYO2LMSGMUB2DMQ, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:10.145084141 (510h9m30.637516596s) [noise=true , address=77.90.31.10:4517 , piece_id=K7OLDFV6YMGHAQL6MQ5Q3MRVHFBZ3D3WHTLUAO2NWG5J5OKBT2LA, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:12.217212852 (510h9m32.7096453s) [noise=true , address=45.91.116.7:4011 , piece_id=33SZJZJMI2KDQFNTYQ46LWT65HFDPSUHLS4FTKW5YTYNV4A52JGQ, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:12.332976431 (510h9m32.825408868s) [noise=true , address=77.90.31.14:4663 , piece_id=YYEMNYCHHYYSCV73TKR2BUX5QJDKJNG43XDBACEKKWW2RVND5J5Q, seg_pos={0 0}, upload=448769]: Done uploading piece. err: <nil> Sep 19 20:17:17.591090950 (510h9m38.083523383s) [noise=true , address=45.91.116.10:4082 , piece_id=3XXBP56ZB4MPVPK4HWJUBDGCLQIVPC3HDVS3Q32WWYP4L3ERMQ4Q, seg_pos={0 0}, upload=448769]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: write tcp 139.59.144.22:39912->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:17:17.591224803 (510h9m38.083657233s) [noise=true , address=45.91.116.14:4191 , piece_id=A7FE4HO4RNWLZXDHQAZLIIF22UIOUYOYKC5C7MCDWOKPW5EJY2SA, seg_pos={0 0}, upload=448769]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:47472->45.91.116.14:4191: use of closed network connection; write tcp 139.59.144.22:47472->45.91.116.14:4191: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47472->45.91.116.14:4191: use of closed network connection Sep 19 20:17:17.591314372 (510h9m38.0837468s) [noise=true , address=77.90.31.19:4841 , piece_id=LVH2Z6VRJV5HE3QZQKXAPFHUDOEXWQT5NTBHXCXFLZVX45MI5AEA, seg_pos={0 0}, upload=448769]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:34458->77.90.31.19:4841: use of closed network connection; write tcp 139.59.144.22:34458->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34458->77.90.31.19:4841: use of closed network connection Sep 19 20:17:17.591437535 (510h9m38.083869962s) [noise=true , address=77.90.31.11:4553 , piece_id=HGUISSOWXVMH2C3YLPKMZA3DDLMFHDV6RP34AEVPNVBFASDS46VA, seg_pos={0 0}, upload=448769]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:57260->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:57260->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57260->77.90.31.11:4553: use of closed network connection Sep 19 20:17:17.591455387 (510h9m38.083887833s) [seg_pos={0 0}, upload=448769]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 19 20:17:17.699249744 (510h9m38.191682203s) [upload=448769]: Done uploading segments... Sep 19 20:17:17.699284160 (510h9m38.191716623s) [upload=448769]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.11040612s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..................................................XXXXXXXXX.] duration: 3.547097939s FAIL (2025-09-19 20:17:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.547126051s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.001439588s FAIL (2025-09-19 20:16:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001468609s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...............................................X..XXXXXXXXX.] duration: 2.824073223s FAIL (2025-09-19 20:16:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== {0 0}, upload=448741]: Done begin upload segment. Sep 19 20:14:58.566523356 (510h7m19.058955785s) [noise=true , address=145.79.227.2:4114 , piece_id=AIHEL25BVV5OGBNU3QSM2IXSD6HZILEKBF37DX6PLLPHW7YFI3DA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.566948030 (510h7m19.059380478s) [noise=true , address=145.79.227.13:4801 , piece_id=PDE3JQI7CSJN335LEPEDQU3JRXH2PMJCKBMDDQUGUECDA7M3JIEA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.567223708 (510h7m19.05965615s) [noise=true , address=45.91.116.5:4299 , piece_id=2SJXGA62AIJLLJ4H6OYPYMBUVZX4NPIKZQAIKXRK233NEKCRJEFQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.567513441 (510h7m19.059945883s) [noise=true , address=145.79.227.14:4839 , piece_id=OSZ2LR6VY3PCIAB2SVZAWG2PYHAWL6XSVUPB6ZH4QWQ3XZQNIYLA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.567781046 (510h7m19.060213488s) [noise=true , address=45.91.116.7:4010 , piece_id=5DHKYM46UJYIFBJCQVD3SOE2ZTFJVGVTVTDLXIGFL3DSZMAWZU4Q, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.568030931 (510h7m19.060463373s) [noise=true , address=145.79.227.6:4257 , piece_id=T2AX54IQY7ZAGU7SWYSMXUF43A3PG3W2P3QF4BBQSGRAW7KEGQPA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.568409091 (510h7m19.060841537s) [noise=true , address=145.79.227.4:4185 , piece_id=4S4XZYDONK4LNFB4Z4R4CSG2VECKY46MX7IYQUI73RU23DLSHHOQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.569022393 (510h7m19.061454838s) [noise=true , address=77.90.31.16:4734 , piece_id=ZXP76MSLMXWG6DRWIKEV6TPWC7JPND3K7XYTDRYBQ4PHM7ACWFAQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.569878375 (510h7m19.06231082s) [noise=true , address=77.90.31.10:4519 , piece_id=VITQIZYERTVGM4QMBEFYDGS66XIX3J5C3C7HOKXZPPKCVM5F2G7Q, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.570396729 (510h7m19.062829174s) [noise=true , address=145.79.227.17:4972 , piece_id=RND375CEIRU3L32JTGRQZL5VFK2W5QUXL4X3BFV2EDQPWDGS3ZHA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.570740593 (510h7m19.063173039s) [noise=true , address=45.91.116.7:4011 , piece_id=DVXQJ2AYXBPU4D3JLB64XLAQYCJ4HCYTOQO4SGETCXVEF2LBDK2A, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.571027753 (510h7m19.063460197s) [noise=true , address=45.91.116.12:4226 , piece_id=EAHZRM6S3SN4VSRHSA44W6HVJOXK6YAJMX24CDMSVMCB7V5W2H4A, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.571644495 (510h7m19.064076939s) [noise=true , address=77.90.31.14:4661 , piece_id=7VKL32ZEB44OSN2UDMRO2JN2SXZVWMNO3WT6TUFOIZ6XOYABJQFQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.571999697 (510h7m19.064432139s) [noise=true , address=45.91.116.6:4975 , piece_id=BDMK7UQZVR4YBPATDQ4D4S72G4WPEODW6V6FBALNB7NGYOAQLTFQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.572293406 (510h7m19.064725857s) [noise=true , address=45.91.116.5:4298 , piece_id=3V7QTOSDJHD2OTKHSXZII3RPKPBENADNTY6EKNJTIZ47GYB35SGQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.572627356 (510h7m19.065059798s) [noise=true , address=77.90.31.15:4698 , piece_id=6RJ7ROINQYMO56AZE7ZRL4XNA7KOPERFPMQYZPLVO2PU2OVEIKPA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:59.200759945 (510h7m19.693192376s) [upload=448741]: Next returned nil segment Sep 19 20:14:59.200780511 (510h7m19.693212956s) [upload=448741]: Waiting for error group managing segments... Sep 19 20:14:59.349992489 (510h7m19.842424948s) [noise=true , address=145.79.227.13:4801 , piece_id=PDE3JQI7CSJN335LEPEDQU3JRXH2PMJCKBMDDQUGUECDA7M3JIEA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.350339605 (510h7m19.842772039s) [noise=true , address=145.79.227.4:4185 , piece_id=4S4XZYDONK4LNFB4Z4R4CSG2VECKY46MX7IYQUI73RU23DLSHHOQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.351108290 (510h7m19.843540719s) [noise=true , address=145.79.227.14:4839 , piece_id=OSZ2LR6VY3PCIAB2SVZAWG2PYHAWL6XSVUPB6ZH4QWQ3XZQNIYLA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.352722314 (510h7m19.845154749s) [noise=true , address=145.79.227.2:4114 , piece_id=AIHEL25BVV5OGBNU3QSM2IXSD6HZILEKBF37DX6PLLPHW7YFI3DA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.364483406 (510h7m19.856915842s) [noise=true , address=145.79.227.6:4257 , piece_id=T2AX54IQY7ZAGU7SWYSMXUF43A3PG3W2P3QF4BBQSGRAW7KEGQPA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.365102880 (510h7m19.857535312s) [noise=true , address=145.79.227.17:4972 , piece_id=RND375CEIRU3L32JTGRQZL5VFK2W5QUXL4X3BFV2EDQPWDGS3ZHA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:15:28.166475644 (510h7m48.658908093s) [noise=true , address=45.91.116.7:4010 , piece_id=5DHKYM46UJYIFBJCQVD3SOE2ZTFJVGVTVTDLXIGFL3DSZMAWZU4Q, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.166557250 (510h7m48.658989694s) [noise=true , address=77.90.31.14:4661 , piece_id=7VKL32ZEB44OSN2UDMRO2JN2SXZVWMNO3WT6TUFOIZ6XOYABJQFQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 139.59.144.22:57338->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.166578773 (510h7m48.659011227s) [noise=true , address=45.91.116.5:4298 , piece_id=3V7QTOSDJHD2OTKHSXZII3RPKPBENADNTY6EKNJTIZ47GYB35SGQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:44394->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.166617156 (510h7m48.659049599s) [noise=true , address=45.91.116.6:4975 , piece_id=BDMK7UQZVR4YBPATDQ4D4S72G4WPEODW6V6FBALNB7NGYOAQLTFQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:55700->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167000072 (510h7m48.659432509s) [noise=true , address=77.90.31.10:4519 , piece_id=VITQIZYERTVGM4QMBEFYDGS66XIX3J5C3C7HOKXZPPKCVM5F2G7Q, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: read tcp 139.59.144.22:59708->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167025978 (510h7m48.659458411s) [noise=true , address=77.90.31.16:4734 , piece_id=ZXP76MSLMXWG6DRWIKEV6TPWC7JPND3K7XYTDRYBQ4PHM7ACWFAQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:48418->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167079087 (510h7m48.659511534s) [noise=true , address=45.91.116.5:4299 , piece_id=2SJXGA62AIJLLJ4H6OYPYMBUVZX4NPIKZQAIKXRK233NEKCRJEFQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:53760->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167093460 (510h7m48.659525904s) [noise=true , address=45.91.116.7:4011 , piece_id=DVXQJ2AYXBPU4D3JLB64XLAQYCJ4HCYTOQO4SGETCXVEF2LBDK2A, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:37346->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167152428 (510h7m48.659584858s) [noise=true , address=45.91.116.12:4226 , piece_id=EAHZRM6S3SN4VSRHSA44W6HVJOXK6YAJMX24CDMSVMCB7V5W2H4A, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:43836->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167179883 (510h7m48.659612313s) [noise=true , address=77.90.31.15:4698 , piece_id=6RJ7ROINQYMO56AZE7ZRL4XNA7KOPERFPMQYZPLVO2PU2OVEIKPA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: read tcp 139.59.144.22:47394->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167201558 (510h7m48.659634011s) [seg_pos={0 0}, upload=448741]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 19 20:15:28.275328209 (510h7m48.76776069s) [upload=448741]: Done uploading segments... Sep 19 20:15:28.275361746 (510h7m48.76779419s) [upload=448741]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.824116711s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 30.000977221s FAIL (2025-09-19 20:15:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001002919s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [................................................X..XXXXXXXXX] duration: 30.110916952s FAIL (2025-09-19 20:15:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== {0 0}, upload=448741]: Done begin upload segment. Sep 19 20:14:58.566523356 (510h7m19.058955785s) [noise=true , address=145.79.227.2:4114 , piece_id=AIHEL25BVV5OGBNU3QSM2IXSD6HZILEKBF37DX6PLLPHW7YFI3DA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.566948030 (510h7m19.059380478s) [noise=true , address=145.79.227.13:4801 , piece_id=PDE3JQI7CSJN335LEPEDQU3JRXH2PMJCKBMDDQUGUECDA7M3JIEA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.567223708 (510h7m19.05965615s) [noise=true , address=45.91.116.5:4299 , piece_id=2SJXGA62AIJLLJ4H6OYPYMBUVZX4NPIKZQAIKXRK233NEKCRJEFQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.567513441 (510h7m19.059945883s) [noise=true , address=145.79.227.14:4839 , piece_id=OSZ2LR6VY3PCIAB2SVZAWG2PYHAWL6XSVUPB6ZH4QWQ3XZQNIYLA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.567781046 (510h7m19.060213488s) [noise=true , address=45.91.116.7:4010 , piece_id=5DHKYM46UJYIFBJCQVD3SOE2ZTFJVGVTVTDLXIGFL3DSZMAWZU4Q, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.568030931 (510h7m19.060463373s) [noise=true , address=145.79.227.6:4257 , piece_id=T2AX54IQY7ZAGU7SWYSMXUF43A3PG3W2P3QF4BBQSGRAW7KEGQPA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.568409091 (510h7m19.060841537s) [noise=true , address=145.79.227.4:4185 , piece_id=4S4XZYDONK4LNFB4Z4R4CSG2VECKY46MX7IYQUI73RU23DLSHHOQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.569022393 (510h7m19.061454838s) [noise=true , address=77.90.31.16:4734 , piece_id=ZXP76MSLMXWG6DRWIKEV6TPWC7JPND3K7XYTDRYBQ4PHM7ACWFAQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.569878375 (510h7m19.06231082s) [noise=true , address=77.90.31.10:4519 , piece_id=VITQIZYERTVGM4QMBEFYDGS66XIX3J5C3C7HOKXZPPKCVM5F2G7Q, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.570396729 (510h7m19.062829174s) [noise=true , address=145.79.227.17:4972 , piece_id=RND375CEIRU3L32JTGRQZL5VFK2W5QUXL4X3BFV2EDQPWDGS3ZHA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.570740593 (510h7m19.063173039s) [noise=true , address=45.91.116.7:4011 , piece_id=DVXQJ2AYXBPU4D3JLB64XLAQYCJ4HCYTOQO4SGETCXVEF2LBDK2A, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.571027753 (510h7m19.063460197s) [noise=true , address=45.91.116.12:4226 , piece_id=EAHZRM6S3SN4VSRHSA44W6HVJOXK6YAJMX24CDMSVMCB7V5W2H4A, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.571644495 (510h7m19.064076939s) [noise=true , address=77.90.31.14:4661 , piece_id=7VKL32ZEB44OSN2UDMRO2JN2SXZVWMNO3WT6TUFOIZ6XOYABJQFQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.571999697 (510h7m19.064432139s) [noise=true , address=45.91.116.6:4975 , piece_id=BDMK7UQZVR4YBPATDQ4D4S72G4WPEODW6V6FBALNB7NGYOAQLTFQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.572293406 (510h7m19.064725857s) [noise=true , address=45.91.116.5:4298 , piece_id=3V7QTOSDJHD2OTKHSXZII3RPKPBENADNTY6EKNJTIZ47GYB35SGQ, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:58.572627356 (510h7m19.065059798s) [noise=true , address=77.90.31.15:4698 , piece_id=6RJ7ROINQYMO56AZE7ZRL4XNA7KOPERFPMQYZPLVO2PU2OVEIKPA, seg_pos={0 0}, upload=448741]: Uploading piece... Sep 19 20:14:59.200759945 (510h7m19.693192376s) [upload=448741]: Next returned nil segment Sep 19 20:14:59.200780511 (510h7m19.693212956s) [upload=448741]: Waiting for error group managing segments... Sep 19 20:14:59.349992489 (510h7m19.842424948s) [noise=true , address=145.79.227.13:4801 , piece_id=PDE3JQI7CSJN335LEPEDQU3JRXH2PMJCKBMDDQUGUECDA7M3JIEA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.350339605 (510h7m19.842772039s) [noise=true , address=145.79.227.4:4185 , piece_id=4S4XZYDONK4LNFB4Z4R4CSG2VECKY46MX7IYQUI73RU23DLSHHOQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.351108290 (510h7m19.843540719s) [noise=true , address=145.79.227.14:4839 , piece_id=OSZ2LR6VY3PCIAB2SVZAWG2PYHAWL6XSVUPB6ZH4QWQ3XZQNIYLA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.352722314 (510h7m19.845154749s) [noise=true , address=145.79.227.2:4114 , piece_id=AIHEL25BVV5OGBNU3QSM2IXSD6HZILEKBF37DX6PLLPHW7YFI3DA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.364483406 (510h7m19.856915842s) [noise=true , address=145.79.227.6:4257 , piece_id=T2AX54IQY7ZAGU7SWYSMXUF43A3PG3W2P3QF4BBQSGRAW7KEGQPA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:14:59.365102880 (510h7m19.857535312s) [noise=true , address=145.79.227.17:4972 , piece_id=RND375CEIRU3L32JTGRQZL5VFK2W5QUXL4X3BFV2EDQPWDGS3ZHA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: <nil> Sep 19 20:15:28.166475644 (510h7m48.658908093s) [noise=true , address=45.91.116.7:4010 , piece_id=5DHKYM46UJYIFBJCQVD3SOE2ZTFJVGVTVTDLXIGFL3DSZMAWZU4Q, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.166557250 (510h7m48.658989694s) [noise=true , address=77.90.31.14:4661 , piece_id=7VKL32ZEB44OSN2UDMRO2JN2SXZVWMNO3WT6TUFOIZ6XOYABJQFQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 139.59.144.22:57338->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.166578773 (510h7m48.659011227s) [noise=true , address=45.91.116.5:4298 , piece_id=3V7QTOSDJHD2OTKHSXZII3RPKPBENADNTY6EKNJTIZ47GYB35SGQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:44394->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.166617156 (510h7m48.659049599s) [noise=true , address=45.91.116.6:4975 , piece_id=BDMK7UQZVR4YBPATDQ4D4S72G4WPEODW6V6FBALNB7NGYOAQLTFQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 139.59.144.22:55700->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167000072 (510h7m48.659432509s) [noise=true , address=77.90.31.10:4519 , piece_id=VITQIZYERTVGM4QMBEFYDGS66XIX3J5C3C7HOKXZPPKCVM5F2G7Q, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: read tcp 139.59.144.22:59708->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167025978 (510h7m48.659458411s) [noise=true , address=77.90.31.16:4734 , piece_id=ZXP76MSLMXWG6DRWIKEV6TPWC7JPND3K7XYTDRYBQ4PHM7ACWFAQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:48418->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167079087 (510h7m48.659511534s) [noise=true , address=45.91.116.5:4299 , piece_id=2SJXGA62AIJLLJ4H6OYPYMBUVZX4NPIKZQAIKXRK233NEKCRJEFQ, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:53760->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167093460 (510h7m48.659525904s) [noise=true , address=45.91.116.7:4011 , piece_id=DVXQJ2AYXBPU4D3JLB64XLAQYCJ4HCYTOQO4SGETCXVEF2LBDK2A, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:37346->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167152428 (510h7m48.659584858s) [noise=true , address=45.91.116.12:4226 , piece_id=EAHZRM6S3SN4VSRHSA44W6HVJOXK6YAJMX24CDMSVMCB7V5W2H4A, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 139.59.144.22:43836->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167179883 (510h7m48.659612313s) [noise=true , address=77.90.31.15:4698 , piece_id=6RJ7ROINQYMO56AZE7ZRL4XNA7KOPERFPMQYZPLVO2PU2OVEIKPA, seg_pos={0 0}, upload=448741]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: read tcp 139.59.144.22:47394->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:28.167201558 (510h7m48.659634011s) [seg_pos={0 0}, upload=448741]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 19 20:15:28.275328209 (510h7m48.76776069s) [upload=448741]: Done uploading segments... Sep 19 20:15:28.275361746 (510h7m48.76779419s) [upload=448741]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.110944641s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 30.002988599s FAIL (2025-09-19 20:14:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.003023528s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................................................X..XXXXXXXX] duration: 30.11563704s FAIL (2025-09-19 20:14:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 19 20:13:57.996371286 (510h6m18.488803729s) [upload=448728]: Starting upload Sep 19 20:13:57.996451040 (510h6m18.488883486s) [upload=448728]: Done starting upload Sep 19 20:13:57.996560495 (510h6m18.488992932s) [upload=448728]: Uploading segments... Sep 19 20:13:57.996815486 (510h6m18.489247919s) [upload=448728]: Got next segment. Inline: false Sep 19 20:13:57.996889650 (510h6m18.4893221s) [upload=448728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:13:57.996897487 (510h6m18.489329929s) [upload=448728]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:13:58.114661784 (510h6m18.60709423s) [seg_pos={0 0}, upload=448728]: Begin upload segment... Sep 19 20:13:58.114730491 (510h6m18.607162923s) [seg_pos={0 0}, upload=448728]: Done begin upload segment. Sep 19 20:13:58.114743467 (510h6m18.607175919s) [noise=true , address=145.79.227.4:4186 , piece_id=3FKBFFMVPWSOGSERMKTBDMYZHDTEJEIDTIQT437JRNCIZHSTSVRA, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.114755668 (510h6m18.60718811s) [noise=true , address=77.90.31.12:4590 , piece_id=UGEQNJMV7NQVRLRO7LHONIJ3HAADCCS3LUNV6DT7NWD6SPFLDKPQ, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.114873169 (510h6m18.607305618s) [noise=true , address=145.79.227.5:4223 , piece_id=YDFJCF6RIFGD2DH7C5ZYMBTUKN6AISDWFFUJONDCWLEPX2LYXGTA, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.115211582 (510h6m18.607644025s) [noise=true , address=45.91.116.11:4118 , piece_id=I5GR5AYZSMNZVAO47TPURQWUNFPGSR3W73CXB557GGTFZSWDK3EQ, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.115557398 (510h6m18.60798984s) [noise=true , address=145.79.227.14:4840 , piece_id=VMJV2HSC3DMGJLNAGPGYRUKKJG5BYE2KVVKHDPBCFDPRF23UMQKQ, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.115920970 (510h6m18.608353417s) [noise=true , address=145.79.227.13:4804 , piece_id=FQLMHIV4RPDP36JKMSWQEG7AUU3NGKDYBOPRFCYQIBMNYS6E6E5Q, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.118144246 (510h6m18.610576698s) [noise=true , address=145.79.227.15:4876 , piece_id=K4CHSCGHXKHRK7UAM4C4SCKV53IIF2MVIGY2GWYCMBSCABLC3JKA, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.118519299 (510h6m18.610951753s) [noise=true , address=45.91.116.7:4011 , piece_id=AV7AUY2H3NAC5MJARIOXKVWSPEO5FVP32DPIJGPW2LISK6LDES3Q, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.118904490 (510h6m18.611336942s) [noise=true , address=45.91.116.14:4190 , piece_id=7ZISPAOUWMBBVBW72OJHWBEXP3JXH4OOR76G4T4AJFF37XDFVURA, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.119287953 (510h6m18.611720405s) [noise=true , address=145.79.227.12:4767 , piece_id=EM36OP6IPMFEAKKWKZU6657VFO6W6JYD5GDXHH6TIC5C3RFHNKVQ, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.119649470 (510h6m18.612081924s) [noise=true , address=77.90.31.17:4771 , piece_id=D6RI325ZVTTXGH5C5TOYWUYW2KJIH6GS4AMUDTX2U7PW5LAP5EQQ, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.120084676 (510h6m18.612517131s) [noise=true , address=145.79.227.19:4078 , piece_id=VWWHM7FITAM2HD4TD7Z24MW2YCVFKN2PRL3IILTFMNXWZ7SAVV4A, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.121701261 (510h6m18.614133711s) [noise=true , address=145.79.227.11:4731 , piece_id=QMUQAQYGQNBQ3YL33PRZDDWQ6T3KCTBYCM7VHFMP34E2PC2GDSCQ, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.122082572 (510h6m18.614515018s) [noise=true , address=77.90.31.17:4770 , piece_id=WDIGTJOK225NXUS527C33HUW3KNX7UKUDXN7IIRE4M3YCGWKMRKQ, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.122416990 (510h6m18.614849437s) [noise=true , address=145.79.227.2:4115 , piece_id=Y47JZKQHJ33YIVYEYA6DBU7YKJ6SRYWBQMSTJRJKQQSVJ7ZU4D3A, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.122730388 (510h6m18.615162835s) [noise=true , address=145.79.227.7:4294 , piece_id=LV63N72QQLYTWM3I6L3XP5TAIXBUAGORRXEAZBQYN76G3AKC22XQ, seg_pos={0 0}, upload=448728]: Uploading piece... Sep 19 20:13:58.744529585 (510h6m19.236962035s) [upload=448728]: Next returned nil segment Sep 19 20:13:58.744550652 (510h6m19.236983095s) [upload=448728]: Waiting for error group managing segments... Sep 19 20:13:58.896581209 (510h6m19.389013651s) [noise=true , address=145.79.227.13:4804 , piece_id=FQLMHIV4RPDP36JKMSWQEG7AUU3NGKDYBOPRFCYQIBMNYS6E6E5Q, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.900246837 (510h6m19.392679284s) [noise=true , address=145.79.227.15:4876 , piece_id=K4CHSCGHXKHRK7UAM4C4SCKV53IIF2MVIGY2GWYCMBSCABLC3JKA, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.900755975 (510h6m19.393188423s) [noise=true , address=145.79.227.12:4767 , piece_id=EM36OP6IPMFEAKKWKZU6657VFO6W6JYD5GDXHH6TIC5C3RFHNKVQ, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.901439473 (510h6m19.3938719s) [noise=true , address=145.79.227.11:4731 , piece_id=QMUQAQYGQNBQ3YL33PRZDDWQ6T3KCTBYCM7VHFMP34E2PC2GDSCQ, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.902477618 (510h6m19.394910054s) [noise=true , address=145.79.227.4:4186 , piece_id=3FKBFFMVPWSOGSERMKTBDMYZHDTEJEIDTIQT437JRNCIZHSTSVRA, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.906683864 (510h6m19.399116304s) [noise=true , address=145.79.227.14:4840 , piece_id=VMJV2HSC3DMGJLNAGPGYRUKKJG5BYE2KVVKHDPBCFDPRF23UMQKQ, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.906881263 (510h6m19.399313692s) [noise=true , address=145.79.227.19:4078 , piece_id=VWWHM7FITAM2HD4TD7Z24MW2YCVFKN2PRL3IILTFMNXWZ7SAVV4A, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.909106521 (510h6m19.40153897s) [noise=true , address=145.79.227.2:4115 , piece_id=Y47JZKQHJ33YIVYEYA6DBU7YKJ6SRYWBQMSTJRJKQQSVJ7ZU4D3A, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.909816400 (510h6m19.402248844s) [noise=true , address=145.79.227.7:4294 , piece_id=LV63N72QQLYTWM3I6L3XP5TAIXBUAGORRXEAZBQYN76G3AKC22XQ, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:13:58.924010220 (510h6m19.416442656s) [noise=true , address=145.79.227.5:4223 , piece_id=YDFJCF6RIFGD2DH7C5ZYMBTUKN6AISDWFFUJONDCWLEPX2LYXGTA, seg_pos={0 0}, upload=448728]: Done uploading piece. err: <nil> Sep 19 20:14:27.679855472 (510h6m48.172287906s) [noise=true , address=45.91.116.7:4011 , piece_id=AV7AUY2H3NAC5MJARIOXKVWSPEO5FVP32DPIJGPW2LISK6LDES3Q, seg_pos={0 0}, upload=448728]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 139.59.144.22:49030->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:27.680042523 (510h6m48.172474959s) [noise=true , address=77.90.31.17:4771 , piece_id=D6RI325ZVTTXGH5C5TOYWUYW2KJIH6GS4AMUDTX2U7PW5LAP5EQQ, seg_pos={0 0}, upload=448728]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: read tcp 139.59.144.22:40708->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:27.680091225 (510h6m48.17252366s) [noise=true , address=45.91.116.14:4190 , piece_id=7ZISPAOUWMBBVBW72OJHWBEXP3JXH4OOR76G4T4AJFF37XDFVURA, seg_pos={0 0}, upload=448728]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 139.59.144.22:44172->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:27.680212935 (510h6m48.172645382s) [noise=true , address=77.90.31.17:4770 , piece_id=WDIGTJOK225NXUS527C33HUW3KNX7UKUDXN7IIRE4M3YCGWKMRKQ, seg_pos={0 0}, upload=448728]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:27.680262648 (510h6m48.172695091s) [noise=true , address=77.90.31.12:4590 , piece_id=UGEQNJMV7NQVRLRO7LHONIJ3HAADCCS3LUNV6DT7NWD6SPFLDKPQ, seg_pos={0 0}, upload=448728]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:50686->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:27.680288295 (510h6m48.172720723s) [noise=true , address=45.91.116.11:4118 , piece_id=I5GR5AYZSMNZVAO47TPURQWUNFPGSR3W73CXB557GGTFZSWDK3EQ, seg_pos={0 0}, upload=448728]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 139.59.144.22:55366->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:27.680320815 (510h6m48.172753243s) [seg_pos={0 0}, upload=448728]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 19 20:14:27.793817538 (510h6m48.286249974s) [upload=448728]: Done uploading segments... Sep 19 20:14:27.793847154 (510h6m48.286279589s) [upload=448728]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.115681339s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.001281562s FAIL (2025-09-19 20:13:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001317096s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..................................................X..XXXXXXX] duration: 30.119945011s FAIL (2025-09-19 20:13:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 48718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:13:08.896382940 (510h5m29.388815388s) [seg_pos={0 0}, upload=448718]: Begin upload segment... Sep 19 20:13:08.896436450 (510h5m29.388868885s) [seg_pos={0 0}, upload=448718]: Done begin upload segment. Sep 19 20:13:08.896481789 (510h5m29.38891424s) [noise=true , address=45.91.116.4:4939 , piece_id=CGJXJKYNK6PX56T6NYJQ3SVYAUEB4BK5OWJQDPJHEYXK2W4UWT5Q, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.896502106 (510h5m29.388934548s) [noise=true , address=145.79.227.19:4079 , piece_id=RY7XKQIM3G4NIUT6FGPKYDDNN6D4WS6NB4YMBQQYA5TWPD5WVQSA, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.896940921 (510h5m29.389373367s) [noise=true , address=77.90.31.13:4628 , piece_id=3IKB7DLH42CIPGQQKI5UE3PJIFJKE7Z4IDUQTKLZCPIPIX4RXLIA, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.896956107 (510h5m29.389388609s) [noise=true , address=145.79.227.13:4801 , piece_id=NJHH7ICNV5SIOAEZKNLHL45FNBLGZYDCZVXZOZNNEWA2LZIYDW4Q, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.897339216 (510h5m29.389771667s) [noise=true , address=45.91.116.7:4010 , piece_id=BSEHFEP56HSISDIVFGSB52PYVQVF5FQPVN7T7UEHOMYREWFAITAQ, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.897367373 (510h5m29.389799823s) [noise=true , address=77.90.31.14:4663 , piece_id=OTMDHPRTYAY7ULFPY6Z2Z3U43AEZHO4PXNG5WEXOYBLVT7KBDMDA, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.897760634 (510h5m29.390193077s) [noise=true , address=45.91.116.6:4973 , piece_id=VW26GP6MHQU5DWWWTYC3KLPIAYJLQOOLZWBUHWS3H3GPLF34TKBA, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.898514012 (510h5m29.390946529s) [noise=true , address=145.79.227.17:4969 , piece_id=35BM46VIKT6SLT36CIQN5NFSSARPR3GNJIUF67NX77QZA3XSVEKA, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.898708598 (510h5m29.391141047s) [noise=true , address=145.79.227.16:4912 , piece_id=E7G25DDY7BYLY3ZXVTZPZSMJSSFM6P2UZ5BXVPE7ENDFMFNWMEGA, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.899020590 (510h5m29.391453035s) [noise=true , address=45.91.116.12:4225 , piece_id=KR24VG5LT346VCPCGYAEVPMT2AEJFSWUPVW7J4NFNMJ7SWODLBTA, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.899117202 (510h5m29.391549654s) [noise=true , address=145.79.227.18:4006 , piece_id=DGH6BKZU6RGCO56BKAZV6WJMXMMYIBCDQQBUHUJLMP2HNGGCENMQ, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.899360743 (510h5m29.391793191s) [noise=true , address=145.79.227.17:4971 , piece_id=H2PVKTEJXGIWO3FDHWLSRRQAWKSNHOSMQ4N6SBOCVDWH4XQ2ZR2Q, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.900446633 (510h5m29.39287909s) [noise=true , address=77.90.31.19:4843 , piece_id=EEX4LUEULR47QE3M55ZH3KKSF64SJBBKQ6KO5KARUKBTMCKTBG6Q, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.901272116 (510h5m29.393704564s) [noise=true , address=45.91.116.8:4045 , piece_id=IADEE3AKI5NRMTECPAWED7IVWEOA5YOCCRGKGXM4L63MSJFDR5YA, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.902235590 (510h5m29.394668042s) [noise=true , address=45.91.116.10:4083 , piece_id=B5UFTVHQUVUVLT3H7EHVJ7TIGEVCMRWSEQRVGB3QXAKF4YJFTZEQ, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:08.903219060 (510h5m29.395651512s) [noise=true , address=145.79.227.4:4187 , piece_id=57XT3YZTCK4OGKCI76VQBES773BFM427AHUSICCDSE2O6LX7ZP6A, seg_pos={0 0}, upload=448718]: Uploading piece... Sep 19 20:13:09.528796590 (510h5m30.021229026s) [upload=448718]: Next returned nil segment Sep 19 20:13:09.529059746 (510h5m30.021492198s) [upload=448718]: Waiting for error group managing segments... Sep 19 20:13:09.672823981 (510h5m30.16525642s) [noise=true , address=145.79.227.16:4912 , piece_id=E7G25DDY7BYLY3ZXVTZPZSMJSSFM6P2UZ5BXVPE7ENDFMFNWMEGA, seg_pos={0 0}, upload=448718]: Done uploading piece. err: <nil> Sep 19 20:13:09.679192287 (510h5m30.171624716s) [noise=true , address=145.79.227.17:4971 , piece_id=H2PVKTEJXGIWO3FDHWLSRRQAWKSNHOSMQ4N6SBOCVDWH4XQ2ZR2Q, seg_pos={0 0}, upload=448718]: Done uploading piece. err: <nil> Sep 19 20:13:09.680145729 (510h5m30.172578157s) [noise=true , address=145.79.227.17:4969 , piece_id=35BM46VIKT6SLT36CIQN5NFSSARPR3GNJIUF67NX77QZA3XSVEKA, seg_pos={0 0}, upload=448718]: Done uploading piece. err: <nil> Sep 19 20:13:09.682872509 (510h5m30.175304955s) [noise=true , address=145.79.227.13:4801 , piece_id=NJHH7ICNV5SIOAEZKNLHL45FNBLGZYDCZVXZOZNNEWA2LZIYDW4Q, seg_pos={0 0}, upload=448718]: Done uploading piece. err: <nil> Sep 19 20:13:09.688402973 (510h5m30.180835409s) [noise=true , address=145.79.227.19:4079 , piece_id=RY7XKQIM3G4NIUT6FGPKYDDNN6D4WS6NB4YMBQQYA5TWPD5WVQSA, seg_pos={0 0}, upload=448718]: Done uploading piece. err: <nil> Sep 19 20:13:09.690661102 (510h5m30.183093536s) [noise=true , address=145.79.227.18:4006 , piece_id=DGH6BKZU6RGCO56BKAZV6WJMXMMYIBCDQQBUHUJLMP2HNGGCENMQ, seg_pos={0 0}, upload=448718]: Done uploading piece. err: <nil> Sep 19 20:13:09.692056224 (510h5m30.184488654s) [noise=true , address=145.79.227.4:4187 , piece_id=57XT3YZTCK4OGKCI76VQBES773BFM427AHUSICCDSE2O6LX7ZP6A, seg_pos={0 0}, upload=448718]: Done uploading piece. err: <nil> Sep 19 20:13:38.433068454 (510h5m58.925500901s) [noise=true , address=45.91.116.6:4973 , piece_id=VW26GP6MHQU5DWWWTYC3KLPIAYJLQOOLZWBUHWS3H3GPLF34TKBA, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:48786->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433329343 (510h5m58.925761771s) [noise=true , address=45.91.116.12:4225 , piece_id=KR24VG5LT346VCPCGYAEVPMT2AEJFSWUPVW7J4NFNMJ7SWODLBTA, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 139.59.144.22:54268->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433372468 (510h5m58.925804897s) [noise=true , address=45.91.116.4:4939 , piece_id=CGJXJKYNK6PX56T6NYJQ3SVYAUEB4BK5OWJQDPJHEYXK2W4UWT5Q, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 139.59.144.22:39146->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433426760 (510h5m58.925859189s) [noise=true , address=77.90.31.14:4663 , piece_id=OTMDHPRTYAY7ULFPY6Z2Z3U43AEZHO4PXNG5WEXOYBLVT7KBDMDA, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 139.59.144.22:60806->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433488559 (510h5m58.925920987s) [noise=true , address=45.91.116.7:4010 , piece_id=BSEHFEP56HSISDIVFGSB52PYVQVF5FQPVN7T7UEHOMYREWFAITAQ, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: read tcp 139.59.144.22:33924->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433533371 (510h5m58.925965799s) [noise=true , address=77.90.31.19:4843 , piece_id=EEX4LUEULR47QE3M55ZH3KKSF64SJBBKQ6KO5KARUKBTMCKTBG6Q, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: read tcp 139.59.144.22:50080->77.90.31.19:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433572241 (510h5m58.926004669s) [noise=true , address=45.91.116.10:4083 , piece_id=B5UFTVHQUVUVLT3H7EHVJ7TIGEVCMRWSEQRVGB3QXAKF4YJFTZEQ, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 139.59.144.22:49740->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433599662 (510h5m58.926032091s) [noise=true , address=45.91.116.8:4045 , piece_id=IADEE3AKI5NRMTECPAWED7IVWEOA5YOCCRGKGXM4L63MSJFDR5YA, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:32852->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433674276 (510h5m58.926106714s) [noise=true , address=77.90.31.13:4628 , piece_id=3IKB7DLH42CIPGQQKI5UE3PJIFJKE7Z4IDUQTKLZCPIPIX4RXLIA, seg_pos={0 0}, upload=448718]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 139.59.144.22:39430->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:38.433732850 (510h5m58.926165362s) [seg_pos={0 0}, upload=448718]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:13:38.551328030 (510h5m59.043760465s) [upload=448718]: Done uploading segments... Sep 19 20:13:38.551355903 (510h5m59.043788345s) [upload=448718]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration:* 30.119963224s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.000808552s FAIL (2025-09-19 20:13:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000821986s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...................................................X..XXXXXX] duration: 30.110163506s FAIL (2025-09-19 20:13:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== UBONYS6UDOVMUYOCMF7FWRH2SI2GUK73Z5FY4P5EQ, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.584128220 (510h4m59.076560672s) [noise=true , address=145.79.227.3:4150 , piece_id=PMWNRSBY2CFLDDKPXSLDV4RKPNI6XMQXVAKLL26HJ45DNXGBIUHQ, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.584932092 (510h4m59.077364537s) [noise=true , address=145.79.227.6:4258 , piece_id=DF2QG5RYCG25IH444QL6XBRBRGVSAZ2OBVWONVAJDVNQ2EAVWSFA, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.585772404 (510h4m59.078204849s) [noise=true , address=77.90.31.10:4519 , piece_id=JZPU2UMYNEYIZ5SFRJOVUYXOJ6Z5ZFJVHTDVU3IUAM27I3GRE5CA, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.587279302 (510h4m59.07971175s) [noise=true , address=45.91.116.10:4082 , piece_id=WAY4SBD3IXAXT237X2CLEME2R3HUPER7S64AA3PTLD2ZTEURQMTA, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.589117493 (510h4m59.081549944s) [noise=true , address=145.79.227.12:4767 , piece_id=LDE2S5W22XLELGVEZOYYLKMVBQGMUEMRMP26YF3TFKEYP3LRZ22A, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.592622884 (510h4m59.085055389s) [noise=true , address=77.90.31.21:4879 , piece_id=VOJU776HGZHWVZPJIVVYPH3S5DHNSMOJKHYQRJ7BO25NA6I4IBUA, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.594934170 (510h4m59.087366622s) [noise=true , address=145.79.227.2:4114 , piece_id=ELJZXCFCGP4ENQDOKX5JJ4V6YUAAT2TN5VUOF5OT2GX5MWA2ZU5A, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.595334858 (510h4m59.087767303s) [noise=true , address=77.90.31.19:4841 , piece_id=5B64U473WSURFOY4CSZOK7QPGN35P3MUXNKZKUFIEHMRS3BK55IA, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.595648897 (510h4m59.088081346s) [noise=true , address=77.90.31.16:4734 , piece_id=AQDZG7GDAR4JUEYP76D2NFBPHLA3TU25PINWIQLULCC6LSEKEPDQ, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.596009198 (510h4m59.088441641s) [noise=true , address=77.90.31.14:4663 , piece_id=PBHXNUG3RJCO7672B5U4KXP7EFXY4IK7HLRYKTKPVXD37LP4RAHA, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.596381235 (510h4m59.088813679s) [noise=true , address=145.79.227.5:4222 , piece_id=BMFFCVNBCYUQHVUQRBI7VEGVXSYX3HGGREOUS4C4NCRKOZMHE4EQ, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.596708009 (510h4m59.089140453s) [noise=true , address=45.91.116.6:4973 , piece_id=CW3TDYLJR2INCVG4KEVVBEJHBZ5LE3EA34R2VLV35WR4BGNBG4FQ, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.597001856 (510h4m59.089434305s) [noise=true , address=45.91.116.4:4940 , piece_id=OALXH533BHWRSV5SJ2BZNY5QVP3Q5LQYGA4JN4DQLEUN5TZNEJJQ, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.597291213 (510h4m59.089723658s) [noise=true , address=77.90.31.19:4842 , piece_id=JUOOOAUG74HYFFPFEP64Z2J2OSJGDRQVTVF34433OSQPECU7UEDQ, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:38.598818907 (510h4m59.091251372s) [noise=true , address=45.91.116.5:4299 , piece_id=P7DLMDGPMHCU6DDQ67XFQHGFXXNIFCM2Z6HVEG24BPCR322AZ5SQ, seg_pos={0 0}, upload=448706]: Uploading piece... Sep 19 20:12:39.240168281 (510h4m59.732600719s) [upload=448706]: Next returned nil segment Sep 19 20:12:39.240190914 (510h4m59.73262338s) [upload=448706]: Waiting for error group managing segments... Sep 19 20:12:39.365790122 (510h4m59.858222551s) [noise=true , address=145.79.227.6:4258 , piece_id=DF2QG5RYCG25IH444QL6XBRBRGVSAZ2OBVWONVAJDVNQ2EAVWSFA, seg_pos={0 0}, upload=448706]: Done uploading piece. err: <nil> Sep 19 20:12:39.371125454 (510h4m59.863557903s) [noise=true , address=145.79.227.3:4150 , piece_id=PMWNRSBY2CFLDDKPXSLDV4RKPNI6XMQXVAKLL26HJ45DNXGBIUHQ, seg_pos={0 0}, upload=448706]: Done uploading piece. err: <nil> Sep 19 20:12:39.402131067 (510h4m59.89456351s) [noise=true , address=145.79.227.2:4114 , piece_id=ELJZXCFCGP4ENQDOKX5JJ4V6YUAAT2TN5VUOF5OT2GX5MWA2ZU5A, seg_pos={0 0}, upload=448706]: Done uploading piece. err: <nil> Sep 19 20:12:39.404746128 (510h4m59.897178561s) [noise=true , address=145.79.227.12:4767 , piece_id=LDE2S5W22XLELGVEZOYYLKMVBQGMUEMRMP26YF3TFKEYP3LRZ22A, seg_pos={0 0}, upload=448706]: Done uploading piece. err: <nil> Sep 19 20:12:39.434064124 (510h4m59.926496553s) [noise=true , address=145.79.227.5:4222 , piece_id=BMFFCVNBCYUQHVUQRBI7VEGVXSYX3HGGREOUS4C4NCRKOZMHE4EQ, seg_pos={0 0}, upload=448706]: Done uploading piece. err: <nil> Sep 19 20:13:08.179043624 (510h5m28.671476064s) [noise=true , address=77.90.31.19:4842 , piece_id=JUOOOAUG74HYFFPFEP64Z2J2OSJGDRQVTVF34433OSQPECU7UEDQ, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179241475 (510h5m28.671673908s) [noise=true , address=77.90.31.21:4879 , piece_id=VOJU776HGZHWVZPJIVVYPH3S5DHNSMOJKHYQRJ7BO25NA6I4IBUA, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 139.59.144.22:53340->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179304974 (510h5m28.671737407s) [noise=true , address=77.90.31.16:4734 , piece_id=AQDZG7GDAR4JUEYP76D2NFBPHLA3TU25PINWIQLULCC6LSEKEPDQ, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 139.59.144.22:40574->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179368654 (510h5m28.671801087s) [noise=true , address=77.90.31.14:4663 , piece_id=PBHXNUG3RJCO7672B5U4KXP7EFXY4IK7HLRYKTKPVXD37LP4RAHA, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 139.59.144.22:44676->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179416064 (510h5m28.67184851s) [noise=true , address=45.91.116.4:4940 , piece_id=OALXH533BHWRSV5SJ2BZNY5QVP3Q5LQYGA4JN4DQLEUN5TZNEJJQ, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: read tcp 139.59.144.22:57388->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179437280 (510h5m28.671869727s) [noise=true , address=77.90.31.10:4519 , piece_id=JZPU2UMYNEYIZ5SFRJOVUYXOJ6Z5ZFJVHTDVU3IUAM27I3GRE5CA, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: context deadline exceeded; context deadline exceeded Sep 19 20:13:08.179488019 (510h5m28.671920457s) [noise=true , address=45.91.116.5:4299 , piece_id=P7DLMDGPMHCU6DDQ67XFQHGFXXNIFCM2Z6HVEG24BPCR322AZ5SQ, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:36882->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179553187 (510h5m28.67198563s) [noise=true , address=45.91.116.6:4973 , piece_id=CW3TDYLJR2INCVG4KEVVBEJHBZ5LE3EA34R2VLV35WR4BGNBG4FQ, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:32842->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179573849 (510h5m28.67200633s) [noise=true , address=77.90.31.15:4697 , piece_id=FJGADNQ7ANRUBONYS6UDOVMUYOCMF7FWRH2SI2GUK73Z5FY4P5EQ, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 139.59.144.22:37556->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179675825 (510h5m28.672108274s) [noise=true , address=45.91.116.10:4082 , piece_id=WAY4SBD3IXAXT237X2CLEME2R3HUPER7S64AA3PTLD2ZTEURQMTA, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 139.59.144.22:38342->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179707623 (510h5m28.672140058s) [noise=true , address=77.90.31.19:4841 , piece_id=5B64U473WSURFOY4CSZOK7QPGN35P3MUXNKZKUFIEHMRS3BK55IA, seg_pos={0 0}, upload=448706]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 139.59.144.22:50328->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:08.179725984 (510h5m28.672158415s) [seg_pos={0 0}, upload=448706]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Sep 19 20:13:08.286647851 (510h5m28.779080284s) [upload=448706]: Done uploading segments... Sep 19 20:13:08.286683557 (510h5m28.779115998s) [upload=448706]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.110183236s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.002043682s FAIL (2025-09-19 20:12:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002058481s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................................X..XXXXX] duration: 30.113486576s FAIL (2025-09-19 20:12:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== ZQ, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.346692860 (510h4m28.839125309s) [noise=true , address=45.91.116.7:4011 , piece_id=UINWF3UWZOH3Q7AGEISCRFXMM572Y6QOV3Y2DPVYNVXZILE3TKUQ, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.346921270 (510h4m28.83935377s) [noise=true , address=145.79.227.6:4259 , piece_id=U7POAGVSAGHFKQNASPQTM7ZOQFSMMMIOSUPHAJVNEP6NRTJBDL3Q, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.347145852 (510h4m28.839578297s) [noise=true , address=77.90.31.10:4520 , piece_id=LEDL4GAOIBWTXWGEE6SYWF5ZZO2WBV3AK3JA2ZUSVUE5SYL3GGMA, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.347370129 (510h4m28.839802573s) [noise=true , address=145.79.227.2:4114 , piece_id=CF6X4D6VAIR7UYDMNS74XUZUQVIAXO6FBW7FSNBZE6HEBI4ZOQLQ, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.347611951 (510h4m28.840044396s) [noise=true , address=77.90.31.19:4843 , piece_id=AW3AACAFNR3RHFUSAMLEVVBXXTEPYMI3OLNF22IMIRN3TIL7RZPQ, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.347771375 (510h4m28.840203804s) [noise=true , address=77.90.31.19:4842 , piece_id=VGBXVRG5JENJFWEB3NZ5JCY426KMFR3PURQA3JADLKEGRSP6Z6AA, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.348058119 (510h4m28.840490568s) [noise=true , address=45.91.116.6:4973 , piece_id=7MATQVDC6BA5LQLXKP3J3AU3P3DWXZGVSCXQU5IO2A6VBG3NEREA, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.348547379 (510h4m28.840979831s) [noise=true , address=45.91.116.12:4225 , piece_id=F3H43OYRVKTQAPAN6RXCPIRVQOORZNEU7IBS4YUV4MNEHK2PODNA, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.348697370 (510h4m28.841129818s) [noise=true , address=45.91.116.8:4048 , piece_id=CBDM7DVAJCZNERR6S6VZ5AVYE5BMRKHYU7CDRP7R64Q4QY7AE4QA, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.349006444 (510h4m28.841438893s) [noise=true , address=45.91.116.4:4940 , piece_id=JGSFIXJQETUPC7TKUZ5HRBMB5PT5OMZ5RNAEJJZYCUSHDK2BLTYQ, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.349306597 (510h4m28.841739045s) [noise=true , address=45.91.116.14:4191 , piece_id=NB4SVPAQ35DJQ524MK46E5NE4NPNDPELBU4OC4HZXU464WLX3XKA, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:08.349463346 (510h4m28.841895795s) [noise=true , address=145.79.227.12:4767 , piece_id=LK4XURBFVB2TBDFZOASSGKRTQF7Y6H6TMGJPF4ORNGONJEGFIEAQ, seg_pos={0 0}, upload=448699]: Uploading piece... Sep 19 20:12:09.002246643 (510h4m29.494679104s) [upload=448699]: Next returned nil segment Sep 19 20:12:09.002270486 (510h4m29.494702952s) [upload=448699]: Waiting for error group managing segments... Sep 19 20:12:09.114711524 (510h4m29.607143961s) [noise=true , address=145.79.227.6:4259 , piece_id=U7POAGVSAGHFKQNASPQTM7ZOQFSMMMIOSUPHAJVNEP6NRTJBDL3Q, seg_pos={0 0}, upload=448699]: Done uploading piece. err: <nil> Sep 19 20:12:09.121165880 (510h4m29.613598309s) [noise=true , address=145.79.227.2:4114 , piece_id=CF6X4D6VAIR7UYDMNS74XUZUQVIAXO6FBW7FSNBZE6HEBI4ZOQLQ, seg_pos={0 0}, upload=448699]: Done uploading piece. err: <nil> Sep 19 20:12:09.129293308 (510h4m29.621725741s) [noise=true , address=145.79.227.12:4767 , piece_id=LK4XURBFVB2TBDFZOASSGKRTQF7Y6H6TMGJPF4ORNGONJEGFIEAQ, seg_pos={0 0}, upload=448699]: Done uploading piece. err: <nil> Sep 19 20:12:37.926154218 (510h4m58.418586668s) [noise=true , address=77.90.31.19:4843 , piece_id=AW3AACAFNR3RHFUSAMLEVVBXXTEPYMI3OLNF22IMIRN3TIL7RZPQ, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: read tcp 139.59.144.22:48428->77.90.31.19:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926573674 (510h4m58.419006102s) [noise=true , address=45.91.116.14:4191 , piece_id=NB4SVPAQ35DJQ524MK46E5NE4NPNDPELBU4OC4HZXU464WLX3XKA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 139.59.144.22:51606->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926677642 (510h4m58.419110096s) [noise=true , address=45.91.116.4:4940 , piece_id=JGSFIXJQETUPC7TKUZ5HRBMB5PT5OMZ5RNAEJJZYCUSHDK2BLTYQ, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 139.59.144.22:54840->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926740599 (510h4m58.419173043s) [noise=true , address=45.91.116.4:4938 , piece_id=B4WXJYZV2WFNB7UVDBSQYXCRVJDELK2YNK6UDBQDAVMEQNLB34HA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 139.59.144.22:56072->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926790200 (510h4m58.419222645s) [noise=true , address=45.91.116.12:4225 , piece_id=F3H43OYRVKTQAPAN6RXCPIRVQOORZNEU7IBS4YUV4MNEHK2PODNA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:44106->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926825159 (510h4m58.419257615s) [noise=true , address=77.90.31.11:4555 , piece_id=TZ2ULHXATWMLZOT6ZNBWJMUEZQLZDNICQLU4DCOCDF5M2U3ZNPZQ, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: read tcp 139.59.144.22:54042->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926866336 (510h4m58.419298768s) [noise=true , address=45.91.116.8:4048 , piece_id=CBDM7DVAJCZNERR6S6VZ5AVYE5BMRKHYU7CDRP7R64Q4QY7AE4QA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 139.59.144.22:56504->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926889814 (510h4m58.419322259s) [noise=true , address=45.91.116.7:4011 , piece_id=UINWF3UWZOH3Q7AGEISCRFXMM572Y6QOV3Y2DPVYNVXZILE3TKUQ, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 139.59.144.22:33566->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926929069 (510h4m58.419361499s) [noise=true , address=77.90.31.16:4733 , piece_id=KF37IM7WITTLSD7LR6YAHZF2URDAAODOW75PMK2WAHWXJBXNFMPA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: read tcp 139.59.144.22:34204->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926952751 (510h4m58.419385197s) [noise=true , address=77.90.31.19:4842 , piece_id=VGBXVRG5JENJFWEB3NZ5JCY426KMFR3PURQA3JADLKEGRSP6Z6AA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 139.59.144.22:55924->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.926989606 (510h4m58.41942206s) [noise=true , address=77.90.31.10:4520 , piece_id=LEDL4GAOIBWTXWGEE6SYWF5ZZO2WBV3AK3JA2ZUSVUE5SYL3GGMA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 139.59.144.22:43928->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.927021642 (510h4m58.419454076s) [noise=true , address=45.91.116.6:4973 , piece_id=7MATQVDC6BA5LQLXKP3J3AU3P3DWXZGVSCXQU5IO2A6VBG3NEREA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 139.59.144.22:55072->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.927031474 (510h4m58.419463921s) [noise=true , address=45.91.116.10:4082 , piece_id=TUTQH4B35XFA7API4JOGKMA2ECYOWZ27PZIIOZFEJ6PEIJCNFFBA, seg_pos={0 0}, upload=448699]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 139.59.144.22:50030->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:37.927052700 (510h4m58.419485152s) [seg_pos={0 0}, upload=448699]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 19 20:12:38.037608458 (510h4m58.530040888s) [upload=448699]: Done uploading segments... Sep 19 20:12:38.037635489 (510h4m58.530067917s) [upload=448699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.11351632s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)