Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.574960185s FAIL (2025-10-06 22:47:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.57506569s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.605683618s FAIL (2025-10-06 22:46:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.605723148s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.797792922s FAIL (2025-10-06 22:46:01 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.79786633s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.638055937s FAIL (2025-10-06 22:45:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.638087781s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.771946692s FAIL (2025-10-06 22:45:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.771994681s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.80139026s FAIL (2025-10-06 22:44:55 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.801415374s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.58332412s FAIL (2025-10-06 22:44:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.583360658s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 3.02221964s FAIL (2025-10-06 22:43:41 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.022244236s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 3.159381265s FAIL (2025-10-06 22:42:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 3.159414552s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.978725817s FAIL (2025-10-06 22:42:34 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.978824908s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.583909066s FAIL (2025-10-06 22:42:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.583937332s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 3.031623617s FAIL (2025-10-06 22:41:57 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.031645088s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.748040793s FAIL (2025-10-06 22:41:12 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.748056186s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.642523097s FAIL (2025-10-06 22:41:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.642554646s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.760880127s FAIL (2025-10-06 22:40:22 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.760961295s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.668858877s FAIL (2025-10-06 22:40:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.668892784s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 3.72197377s FAIL (2025-10-06 22:40:08 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.722006053s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.622651489s FAIL (2025-10-06 22:39:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.622683993s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.947481655s FAIL (2025-10-06 22:39:26 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.947506425s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.563854399s FAIL (2025-10-06 22:38:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.563885027s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.857320678s FAIL (2025-10-06 22:38:29 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.857341637s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.572256154s FAIL (2025-10-06 22:37:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.57231739s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 3.010640226s FAIL (2025-10-06 22:37:51 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.010667451s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 3.110249013s FAIL (2025-10-06 22:37:09 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.110271988s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.469645077s FAIL (2025-10-06 22:37:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.469681795s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.72830304s FAIL (2025-10-06 22:36:07 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.728328138s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.688912378s FAIL (2025-10-06 22:35:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.689080055s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.92775488s FAIL (2025-10-06 22:35:14 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.927784836s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 3.195625798s FAIL (2025-10-06 22:35:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 3.195663668s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.594979864s FAIL (2025-10-06 22:34:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.595009087s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.845911134s FAIL (2025-10-06 22:34:04 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.846023778s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.667031781s FAIL (2025-10-06 22:33:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.667140178s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.86989874s FAIL (2025-10-06 22:33:28 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.869928131s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.574241827s FAIL (2025-10-06 22:33:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.574276535s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.995497577s FAIL (2025-10-06 22:32:46 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.995558963s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.801446399s FAIL (2025-10-06 22:32:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.801686459s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 3.052662546s FAIL (2025-10-06 22:31:43 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.052694155s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.919345319s FAIL (2025-10-06 22:31:21 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.919819185s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.590493472s FAIL (2025-10-06 22:31:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.590530345s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.713883901s FAIL (2025-10-06 22:30:26 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.713910064s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.57788508s FAIL (2025-10-06 22:30:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.577924008s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 3.541136276s FAIL (2025-10-06 22:30:05 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.542313808s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 3.649646006s FAIL (2025-10-06 22:29:33 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.649671674s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.507651529s FAIL (2025-10-06 22:29:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.507678652s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 3.21596377s FAIL (2025-10-06 22:28:54 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.215989867s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.77794398s FAIL (2025-10-06 22:28:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.777973312s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 3.386375137s FAIL (2025-10-06 22:28:30 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.386873463s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.579688945s FAIL (2025-10-06 22:28:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.57973651s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.615222477s FAIL (2025-10-06 22:27:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 6 22:04:40.124443389 (320h34m27.927907966s) [upload=282109]: Starting upload Oct 6 22:04:40.124539230 (320h34m27.928003824s) [upload=282109]: Done starting upload Oct 6 22:04:40.124671777 (320h34m27.928136367s) [upload=282109]: Uploading segments... Oct 6 22:04:40.124928300 (320h34m27.928392877s) [upload=282109]: Got next segment. Inline: false Oct 6 22:04:40.125013547 (320h34m27.928478124s) [upload=282109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 22:04:40.125018228 (320h34m27.928482805s) [upload=282109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 22:04:40.240598039 (320h34m28.044062631s) [seg_pos={0 0}, upload=282109]: Begin upload segment... Oct 6 22:04:40.240655734 (320h34m28.044120316s) [seg_pos={0 0}, upload=282109]: Done begin upload segment. Oct 6 22:04:40.240700158 (320h34m28.044164749s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.240706532 (320h34m28.044171132s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241368237 (320h34m28.044832832s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.241657518 (320h34m28.045122113s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242196456 (320h34m28.045661058s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242470596 (320h34m28.045935177s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.242870747 (320h34m28.046335346s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243396833 (320h34m28.046861427s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243572746 (320h34m28.04703734s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.243925943 (320h34m28.047390536s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244076216 (320h34m28.047540797s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244531195 (320h34m28.047995789s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.244754757 (320h34m28.04821936s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245254511 (320h34m28.048719092s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.245936892 (320h34m28.049401494s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.246772240 (320h34m28.050236836s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Uploading piece... Oct 6 22:04:40.897188204 (320h34m28.700652796s) [upload=282109]: Next returned nil segment Oct 6 22:04:40.897219546 (320h34m28.700684136s) [upload=282109]: Waiting for error group managing segments... Oct 6 22:04:41.048159492 (320h34m28.851624076s) [noise=true , address=145.79.227.7:4295 , piece_id=UAA3LG42BPSKNPG4AU77YOYFFXOFKULVLAJ6XZUJKE2S23Z657XA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.049015249 (320h34m28.85247983s) [noise=true , address=145.79.227.14:4837 , piece_id=5IDBA7YT72YGMORIXN6MTHFDEGIGAZQKJPK7BVTJXUIXZRPYM5ZQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.050414372 (320h34m28.853878955s) [noise=true , address=145.79.227.17:4970 , piece_id=AXBKH6PC262KSHI4JMDT6YY5K5TTJAIPGXFFIQANP2WBSGOCQFNQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051320052 (320h34m28.854784631s) [noise=true , address=145.79.227.15:4874 , piece_id=BQDGUCN5LG4Z7EOZMWJ7ZN4ETT66TDFDZMLKLSHUWT6XKU2AG3LA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.051571274 (320h34m28.855035852s) [noise=true , address=145.79.227.5:4223 , piece_id=W7IM5CKGIZUCEL4J5AFCJXDU3F7MQOW2OQ235GGABXC6WNA6U32A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.053376635 (320h34m28.856841217s) [noise=true , address=145.79.227.13:4804 , piece_id=63ZS7UUHAGOFLXVWSNHKMYZW3XS7MBOA3RWX2PNC3XGXF66SQOWA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.058058491 (320h34m28.861523083s) [noise=true , address=145.79.227.4:4185 , piece_id=NINUN52EB6SXBRVQ7H3IDEDLRWULNA5JVT3PSHUPT3QP4UDT3AEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.061265712 (320h34m28.864730303s) [noise=true , address=145.79.227.14:4838 , piece_id=VYJEY3NIZ4A7OHWFBQ32V3I5FCDQPTN5BKFKPJY5YODCO6VR2LKQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.068942363 (320h34m28.872406944s) [noise=true , address=145.79.227.7:4293 , piece_id=T2RJRAYDI6MSKYPYX4GFGYPHGT6NWKAOKTMKM5VQZ754QQRG246A, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:04:41.069059755 (320h34m28.872524339s) [noise=true , address=145.79.227.11:4729 , piece_id=DMJI6AY7QTHFQNACHSNILPSPF6VHU2J3FYLYXKT2T3WQDJX2G2UA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.625853604 (320h34m54.429318188s) [noise=true , address=45.91.116.13:4153 , piece_id=CQOSM6U7D2QHNEJXAXTWAKGBU2TA2FXJRWMWOTWKS5C7IWOSYUPA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:06.881003880 (320h34m54.684468458s) [noise=true , address=77.90.31.17:4770 , piece_id=3PZ26VTEFYO4QPS55GCWQVIDHJ335IYUCZ2LMAZZJ6WS63HYECUA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: <nil> Oct 6 22:05:09.828604831 (320h34m57.632069414s) [noise=true , address=45.91.116.5:4299 , piece_id=K7B6HEYCWL727OQKWLI74BDPSUSYF3J7AO343F2JAEI33KHMBT3Q, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 139.59.144.22:43970->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828746252 (320h34m57.632210833s) [noise=true , address=45.91.116.9:4263 , piece_id=YSEBB5FS2452GPIZMEYB2SOVJSPCN4VCP7VTPD6TYMFCLHUSSQEQ, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 139.59.144.22:33152->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828787351 (320h34m57.632251935s) [noise=true , address=77.90.31.16:4733 , piece_id=EAURYAED4RXZLSPBWK2AKTIF7ZZE5XMQVSNDO4MWE3QTH3KXDRHA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828829118 (320h34m57.632293717s) [noise=true , address=77.90.31.12:4590 , piece_id=N7ALXLS33FQB5LU23F5D4LB4LLRAI32NIETJ2RJMOHOCMSKFM4CA, seg_pos={0 0}, upload=282109]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:41962->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 22:05:09.828883864 (320h34m57.632348456s) [seg_pos={0 0}, upload=282109]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 6 22:05:09.936174235 (320h34m57.73963882s) [upload=282109]: Done uploading segments... Oct 6 22:05:09.936254043 (320h34m57.739718636s) [upload=282109]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.615257425s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.978244873s FAIL (2025-10-06 22:27:17 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.978329963s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)