Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [......X......X..XXXX.........................XX..X..X......X] duration: 30.001019383s FAIL (2025-09-22 04:26:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001046376s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [..........X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.......] duration: 2.520469673s FAIL (2025-09-22 04:25:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.520534824s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...........X..X..XX..X......XXX.....XX.X..X.XX..X..X.X......] duration: 2.491683538s FAIL (2025-09-22 04:25:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.491718581s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X......X..XXXX.........................XX..X..X......] duration: 2.911280385s FAIL (2025-09-22 04:25:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.911337904s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [............X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.....] duration: 2.446345365s FAIL (2025-09-22 04:24:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.446382278s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X.......X......X..XXXX.........................XX..X..X.....] duration: 2.798440556s FAIL (2025-09-22 04:24:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.798480152s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.............X..X..XX..X......XXX.....XX.X..X.XX..X..X.X....] duration: 2.411862721s FAIL (2025-09-22 04:23:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.411895631s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X.......X......X..XXXX.........................XX..X..X....] duration: 2.667542488s FAIL (2025-09-22 04:23:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.667569992s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X.......X......X..XXXX.........................XX..X..X...] duration: 2.809985361s FAIL (2025-09-22 04:22:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.810025386s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [..............X..X..XX..X......XXX.....XX.X..X.XX..X..X.X...] duration: 2.576557438s FAIL (2025-09-22 04:22:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.576628574s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...............X..X..XX..X......XXX.....XX.X..X.XX..X..X.X..] duration: 2.442599624s FAIL (2025-09-22 04:21:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.442634421s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X.......X......X..XXXX.........................XX..X..X..] duration: 2.622034142s FAIL (2025-09-22 04:21:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.622085983s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [................X..X..XX..X......XXX.....XX.X..X.XX..X..X.X.] duration: 2.384778733s FAIL (2025-09-22 04:20:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.384823505s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.................X..X..XX..X......XXX.....XX.X..X.XX..X..X.X] duration: 30.110941553s FAIL (2025-09-22 04:20:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:20:25.267832059 (566h12m45.760264491s) [upload=498115]: Starting upload Sep 22 04:20:25.267913906 (566h12m45.760346343s) [upload=498115]: Done starting upload Sep 22 04:20:25.268155840 (566h12m45.760588274s) [upload=498115]: Uploading segments... Sep 22 04:20:25.268434238 (566h12m45.760866674s) [upload=498115]: Got next segment. Inline: false Sep 22 04:20:25.268526154 (566h12m45.760958589s) [upload=498115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:20:25.268533661 (566h12m45.760966124s) [upload=498115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:20:25.385146545 (566h12m45.877578986s) [seg_pos={0 0}, upload=498115]: Begin upload segment... Sep 22 04:20:25.385197224 (566h12m45.877629653s) [seg_pos={0 0}, upload=498115]: Done begin upload segment. Sep 22 04:20:25.385225061 (566h12m45.877657507s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385259555 (566h12m45.877692007s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385580251 (566h12m45.878012708s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.385961294 (566h12m45.87839374s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386422953 (566h12m45.878855402s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386492567 (566h12m45.878925021s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.386825686 (566h12m45.87925813s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387102239 (566h12m45.879534684s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387404982 (566h12m45.879837429s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387540348 (566h12m45.879972793s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.387949155 (566h12m45.880381609s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388326089 (566h12m45.880758553s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388663859 (566h12m45.881096311s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.388753731 (566h12m45.881186179s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389081946 (566h12m45.881514389s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.389379226 (566h12m45.881811669s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Uploading piece... Sep 22 04:20:25.990863597 (566h12m46.483296025s) [upload=498115]: Next returned nil segment Sep 22 04:20:25.990881639 (566h12m46.48331408s) [upload=498115]: Waiting for error group managing segments... Sep 22 04:20:26.130614295 (566h12m46.623046727s) [noise=true , address=45.91.116.4:4938 , piece_id=MV73YRITAXW32JTKJ5AF7UC2BX5Y5SGDPOB7WSWA55LC2U7AOSRA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132016117 (566h12m46.624448549s) [noise=true , address=77.90.31.16:4735 , piece_id=B3PXGZL5AD5BSDVBQ75XIYIEA6OGKUIGUE5NVDT35WN2W3HLGVVQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132373859 (566h12m46.624806303s) [noise=true , address=45.91.116.10:4083 , piece_id=4C6VNH54UECZQ53VZYEHF2WZQIFS5J3PYPE7JTDJN2AKB2MOLCNA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.132531076 (566h12m46.624963504s) [noise=true , address=77.90.31.14:4662 , piece_id=UGLT4JHX6WPWY2VMJLBGJQAUPHF3AA4QRPIYZSUA46EFHIXO5QBA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.133649560 (566h12m46.626081992s) [noise=true , address=45.91.116.13:4154 , piece_id=ZR4TWFII5SP5G3FQH36UAGLAJJ3E36SQTGOH2TZFFZFAWLY5FDOA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.134739518 (566h12m46.627171955s) [noise=true , address=77.90.31.12:4589 , piece_id=ZD3CQS4V7QFKYCEOBYI2ZNDNHBQUEIFPKBKKM6SZWD77TEWDZBPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.135235002 (566h12m46.627667438s) [noise=true , address=45.91.116.6:4975 , piece_id=PYI7HRREQRVIUJN4AOEGNFFCU7QD6S4SA2CCDVHVURJGDZQKRSJQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.139057204 (566h12m46.631489636s) [noise=true , address=45.91.116.4:4940 , piece_id=MH3DLHPLDFV5H45EWIKYOMLHO6SKSWMTOXINAQE5OVXB46F3TU4A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.150084683 (566h12m46.642517117s) [noise=true , address=77.90.31.15:4698 , piece_id=RWIUYPSVG76OTK4PSOFGI452QK37TF2TKCS3MZQ6HBMKUXRSDUCQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.153229156 (566h12m46.645661591s) [noise=true , address=77.90.31.14:4663 , piece_id=MUZ4DFYDMJP5QWN5OTNXB7ZEO3W6TLEHGAE4RSQGQHJGZRN7WNKQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156966637 (566h12m46.649399069s) [noise=true , address=77.90.31.21:4879 , piece_id=SDRQGGQ7HW6WJFAGR24R6F4K3B36BS3L7GG7TGUG6GRRIWPH7U2Q, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:26.156982994 (566h12m46.649415423s) [noise=true , address=45.91.116.11:4118 , piece_id=62BDSJDMJPCMXKOUFQGH5HFQMUHHY2P6YUALV5J6KVI7IL6JCOBQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: <nil> Sep 22 04:20:54.962276597 (566h13m15.454709039s) [noise=true , address=145.79.227.17:4971 , piece_id=525PT6S5WHU4WGT7E2KUCPPSNSKCAHIBXR2ONPESPTPZBLAUIOPA, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:20:54.962491300 (566h13m15.45492375s) [noise=true , address=145.79.227.19:4078 , piece_id=A3J4SVD6RL35SX7UA7KZIVCJWUB7COZDJACL5VB7HBEJF4OBLJ5A, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 04:20:54.962561914 (566h13m15.454994349s) [noise=true , address=145.79.227.4:4187 , piece_id=SEVVK2W5SNFZ7RQZANRWZYAD6Q6G6VCA3OBCAAIE64PJ3HXSU3PQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; context deadline exceeded Sep 22 04:20:54.962625206 (566h13m15.455057652s) [noise=true , address=145.79.227.11:4731 , piece_id=ZL5LKUJH47HLM4QUX4IMFD6J6HILXENMEEGTYUUWCRG5TJCBZITQ, seg_pos={0 0}, upload=498115]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:20:54.962699426 (566h13m15.455131877s) [seg_pos={0 0}, upload=498115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:20:55.071381920 (566h13m15.563814385s) [upload=498115]: Done uploading segments... Sep 22 04:20:55.071433178 (566h13m15.563865628s) [upload=498115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.110969708s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....X.......X......X..XXXX.........................XX..X..X.] duration: 2.811961215s FAIL (2025-09-22 04:20:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.811983697s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....X.......X......X..XXXX.........................XX..X..X] duration: 30.000603964s FAIL (2025-09-22 04:20:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000623512s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..................X..X..XX..X......XXX.....XX.X..X.XX..X..X.] duration: 2.374293327s FAIL (2025-09-22 04:19:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:18:40.439175093 (566h11m0.931607525s) [upload=498096]: Starting upload Sep 22 04:18:40.439292910 (566h11m0.931725356s) [upload=498096]: Done starting upload Sep 22 04:18:40.439375187 (566h11m0.931807622s) [upload=498096]: Uploading segments... Sep 22 04:18:40.439723359 (566h11m0.932155811s) [upload=498096]: Got next segment. Inline: false Sep 22 04:18:40.439833093 (566h11m0.932265543s) [upload=498096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:18:40.439841572 (566h11m0.932274021s) [upload=498096]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:18:40.554509146 (566h11m1.046941598s) [seg_pos={0 0}, upload=498096]: Begin upload segment... Sep 22 04:18:40.554584424 (566h11m1.047016871s) [seg_pos={0 0}, upload=498096]: Done begin upload segment. Sep 22 04:18:40.554616958 (566h11m1.047049437s) [noise=true , address=145.79.227.13:4803 , piece_id=6HR7IOH2S3KOI6FE5M4VJVXIHLEILIAFS7NVURLBM2UZC533NEYQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.554636370 (566h11m1.047068821s) [noise=true , address=77.90.31.12:4592 , piece_id=CAL3VAEZHZRBGJVNWNVJXIWQD4WXWMTKY7PELLVJBIIPTSXJU42A, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.555175363 (566h11m1.047607839s) [noise=true , address=145.79.227.6:4259 , piece_id=H4R5MP67R3PRCWFUWOBSOMYFHYUHDSHJLQEIUEYR4EY6R52JTNJQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.555211121 (566h11m1.047643577s) [noise=true , address=145.79.227.12:4767 , piece_id=52KI3VLSGHGJOKYPKBQLXHJZMCJ6HDVPKT7DYW2O4QISRPGTCJIA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.555668393 (566h11m1.048100846s) [noise=true , address=77.90.31.21:4877 , piece_id=ILYDJR7MFKIWVVRGA43FR6RO2LZX7C5K3W23Q2WSK6ZMPZOLGTVA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.555737690 (566h11m1.048170143s) [noise=true , address=45.91.116.13:4155 , piece_id=K36OXFO5KYKMUZ2YMXKB6BA73L27E7LMNEXKPND6VJJETJUXYHLA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.556268212 (566h11m1.048700661s) [noise=true , address=45.91.116.14:4190 , piece_id=YI3OYG6O65E5IU5XJOXXRXTTLMK2GWPTVHHHPX3MHPZC4UQL7KOA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.556299480 (566h11m1.048731934s) [noise=true , address=145.79.227.17:4972 , piece_id=ABMDOPTPGZMGXZD3S2DLBCZZHHVBLQKAPNK2KVAU7WYKJTMRYSWA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.556662197 (566h11m1.04909464s) [noise=true , address=45.91.116.9:4261 , piece_id=2JGYN2JOHHBZVMRBIJDC7ACERN2QOJEM45YBGXDFTPCKNWNCPNGQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.556877347 (566h11m1.049309776s) [noise=true , address=145.79.227.13:4801 , piece_id=Q4MHR3OPGVAUNQR5ZZNMQDJPG77EPRV4HRNBVQFUR6E2NLPLVK2A, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.557066207 (566h11m1.04949865s) [noise=true , address=145.79.227.11:4730 , piece_id=CPTOYO4DM6CHTC4VGDQZ4AH53Z7PK7KU3VRQPKXXL53QZ75MFEQQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.557422220 (566h11m1.049854671s) [noise=true , address=77.90.31.11:4555 , piece_id=F4S7RIS46F2C5WNN7U3PUPB2ZDHKGTPK4PSQWBY3NYRPYE75BZZQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.557963287 (566h11m1.050395735s) [noise=true , address=45.91.116.6:4973 , piece_id=OQ6SGUTPT4YDLCH42LDZHOFLIB5BXKKIE7AZKAXHHOHHABQEMDUQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.558331298 (566h11m1.050763743s) [noise=true , address=77.90.31.19:4842 , piece_id=SNHVHFMHNDOFRIBUFN57FMB6WW5M4F6ZSYKVKHTTIP7PHWIXFKQA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.559298014 (566h11m1.051730463s) [noise=true , address=77.90.31.17:4771 , piece_id=TVY5SS3RKYSZVM3XO4M773NRJV2A2NGHJ7HDEMFR4PWVRBPBTA5A, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.559835503 (566h11m1.052267955s) [noise=true , address=77.90.31.15:4697 , piece_id=7PHDYIDCN6DH6UDFG7BRD4YFID3M3QQI6BOCZ77RNX5SFFTOPAKA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:41.172763834 (566h11m1.665196269s) [upload=498096]: Next returned nil segment Sep 22 04:18:41.172784331 (566h11m1.665216786s) [upload=498096]: Waiting for error group managing segments... Sep 22 04:18:41.307975282 (566h11m1.800407715s) [noise=true , address=45.91.116.9:4261 , piece_id=2JGYN2JOHHBZVMRBIJDC7ACERN2QOJEM45YBGXDFTPCKNWNCPNGQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.308577569 (566h11m1.801010018s) [noise=true , address=77.90.31.11:4555 , piece_id=F4S7RIS46F2C5WNN7U3PUPB2ZDHKGTPK4PSQWBY3NYRPYE75BZZQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.309133229 (566h11m1.801565673s) [noise=true , address=77.90.31.19:4842 , piece_id=SNHVHFMHNDOFRIBUFN57FMB6WW5M4F6ZSYKVKHTTIP7PHWIXFKQA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.314939261 (566h11m1.80737169s) [noise=true , address=45.91.116.13:4155 , piece_id=K36OXFO5KYKMUZ2YMXKB6BA73L27E7LMNEXKPND6VJJETJUXYHLA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.316400192 (566h11m1.808832623s) [noise=true , address=77.90.31.17:4771 , piece_id=TVY5SS3RKYSZVM3XO4M773NRJV2A2NGHJ7HDEMFR4PWVRBPBTA5A, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.319177316 (566h11m1.811609759s) [noise=true , address=77.90.31.15:4697 , piece_id=7PHDYIDCN6DH6UDFG7BRD4YFID3M3QQI6BOCZ77RNX5SFFTOPAKA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.320342285 (566h11m1.812774714s) [noise=true , address=77.90.31.21:4877 , piece_id=ILYDJR7MFKIWVVRGA43FR6RO2LZX7C5K3W23Q2WSK6ZMPZOLGTVA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.321748835 (566h11m1.814181264s) [noise=true , address=45.91.116.14:4190 , piece_id=YI3OYG6O65E5IU5XJOXXRXTTLMK2GWPTVHHHPX3MHPZC4UQL7KOA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.321987071 (566h11m1.814419516s) [noise=true , address=45.91.116.6:4973 , piece_id=OQ6SGUTPT4YDLCH42LDZHOFLIB5BXKKIE7AZKAXHHOHHABQEMDUQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.325593759 (566h11m1.818026188s) [noise=true , address=77.90.31.12:4592 , piece_id=CAL3VAEZHZRBGJVNWNVJXIWQD4WXWMTKY7PELLVJBIIPTSXJU42A, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:19:10.150545639 (566h11m30.642978073s) [noise=true , address=145.79.227.11:4730 , piece_id=CPTOYO4DM6CHTC4VGDQZ4AH53Z7PK7KU3VRQPKXXL53QZ75MFEQQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150606746 (566h11m30.643039178s) [noise=true , address=145.79.227.6:4259 , piece_id=H4R5MP67R3PRCWFUWOBSOMYFHYUHDSHJLQEIUEYR4EY6R52JTNJQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150655454 (566h11m30.643087895s) [noise=true , address=145.79.227.17:4972 , piece_id=ABMDOPTPGZMGXZD3S2DLBCZZHHVBLQKAPNK2KVAU7WYKJTMRYSWA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150718539 (566h11m30.643150968s) [noise=true , address=145.79.227.13:4803 , piece_id=6HR7IOH2S3KOI6FE5M4VJVXIHLEILIAFS7NVURLBM2UZC533NEYQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150780577 (566h11m30.643213015s) [noise=true , address=145.79.227.12:4767 , piece_id=52KI3VLSGHGJOKYPKBQLXHJZMCJ6HDVPKT7DYW2O4QISRPGTCJIA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150825180 (566h11m30.643257627s) [noise=true , address=145.79.227.13:4801 , piece_id=Q4MHR3OPGVAUNQR5ZZNMQDJPG77EPRV4HRNBVQFUR6E2NLPLVK2A, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150857541 (566h11m30.643289987s) [seg_pos={0 0}, upload=498096]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:19:10.256585180 (566h11m30.749017615s) [upload=498096]: Done uploading segments... Sep 22 04:19:10.256612970 (566h11m30.749045406s) [upload=498096]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.374400213s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [X..................X..X..XX..X......XXX.....XX.X..X.XX..X..X] duration: 30.107770199s FAIL (2025-09-22 04:19:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:18:40.439175093 (566h11m0.931607525s) [upload=498096]: Starting upload Sep 22 04:18:40.439292910 (566h11m0.931725356s) [upload=498096]: Done starting upload Sep 22 04:18:40.439375187 (566h11m0.931807622s) [upload=498096]: Uploading segments... Sep 22 04:18:40.439723359 (566h11m0.932155811s) [upload=498096]: Got next segment. Inline: false Sep 22 04:18:40.439833093 (566h11m0.932265543s) [upload=498096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:18:40.439841572 (566h11m0.932274021s) [upload=498096]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:18:40.554509146 (566h11m1.046941598s) [seg_pos={0 0}, upload=498096]: Begin upload segment... Sep 22 04:18:40.554584424 (566h11m1.047016871s) [seg_pos={0 0}, upload=498096]: Done begin upload segment. Sep 22 04:18:40.554616958 (566h11m1.047049437s) [noise=true , address=145.79.227.13:4803 , piece_id=6HR7IOH2S3KOI6FE5M4VJVXIHLEILIAFS7NVURLBM2UZC533NEYQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.554636370 (566h11m1.047068821s) [noise=true , address=77.90.31.12:4592 , piece_id=CAL3VAEZHZRBGJVNWNVJXIWQD4WXWMTKY7PELLVJBIIPTSXJU42A, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.555175363 (566h11m1.047607839s) [noise=true , address=145.79.227.6:4259 , piece_id=H4R5MP67R3PRCWFUWOBSOMYFHYUHDSHJLQEIUEYR4EY6R52JTNJQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.555211121 (566h11m1.047643577s) [noise=true , address=145.79.227.12:4767 , piece_id=52KI3VLSGHGJOKYPKBQLXHJZMCJ6HDVPKT7DYW2O4QISRPGTCJIA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.555668393 (566h11m1.048100846s) [noise=true , address=77.90.31.21:4877 , piece_id=ILYDJR7MFKIWVVRGA43FR6RO2LZX7C5K3W23Q2WSK6ZMPZOLGTVA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.555737690 (566h11m1.048170143s) [noise=true , address=45.91.116.13:4155 , piece_id=K36OXFO5KYKMUZ2YMXKB6BA73L27E7LMNEXKPND6VJJETJUXYHLA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.556268212 (566h11m1.048700661s) [noise=true , address=45.91.116.14:4190 , piece_id=YI3OYG6O65E5IU5XJOXXRXTTLMK2GWPTVHHHPX3MHPZC4UQL7KOA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.556299480 (566h11m1.048731934s) [noise=true , address=145.79.227.17:4972 , piece_id=ABMDOPTPGZMGXZD3S2DLBCZZHHVBLQKAPNK2KVAU7WYKJTMRYSWA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.556662197 (566h11m1.04909464s) [noise=true , address=45.91.116.9:4261 , piece_id=2JGYN2JOHHBZVMRBIJDC7ACERN2QOJEM45YBGXDFTPCKNWNCPNGQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.556877347 (566h11m1.049309776s) [noise=true , address=145.79.227.13:4801 , piece_id=Q4MHR3OPGVAUNQR5ZZNMQDJPG77EPRV4HRNBVQFUR6E2NLPLVK2A, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.557066207 (566h11m1.04949865s) [noise=true , address=145.79.227.11:4730 , piece_id=CPTOYO4DM6CHTC4VGDQZ4AH53Z7PK7KU3VRQPKXXL53QZ75MFEQQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.557422220 (566h11m1.049854671s) [noise=true , address=77.90.31.11:4555 , piece_id=F4S7RIS46F2C5WNN7U3PUPB2ZDHKGTPK4PSQWBY3NYRPYE75BZZQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.557963287 (566h11m1.050395735s) [noise=true , address=45.91.116.6:4973 , piece_id=OQ6SGUTPT4YDLCH42LDZHOFLIB5BXKKIE7AZKAXHHOHHABQEMDUQ, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.558331298 (566h11m1.050763743s) [noise=true , address=77.90.31.19:4842 , piece_id=SNHVHFMHNDOFRIBUFN57FMB6WW5M4F6ZSYKVKHTTIP7PHWIXFKQA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.559298014 (566h11m1.051730463s) [noise=true , address=77.90.31.17:4771 , piece_id=TVY5SS3RKYSZVM3XO4M773NRJV2A2NGHJ7HDEMFR4PWVRBPBTA5A, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:40.559835503 (566h11m1.052267955s) [noise=true , address=77.90.31.15:4697 , piece_id=7PHDYIDCN6DH6UDFG7BRD4YFID3M3QQI6BOCZ77RNX5SFFTOPAKA, seg_pos={0 0}, upload=498096]: Uploading piece... Sep 22 04:18:41.172763834 (566h11m1.665196269s) [upload=498096]: Next returned nil segment Sep 22 04:18:41.172784331 (566h11m1.665216786s) [upload=498096]: Waiting for error group managing segments... Sep 22 04:18:41.307975282 (566h11m1.800407715s) [noise=true , address=45.91.116.9:4261 , piece_id=2JGYN2JOHHBZVMRBIJDC7ACERN2QOJEM45YBGXDFTPCKNWNCPNGQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.308577569 (566h11m1.801010018s) [noise=true , address=77.90.31.11:4555 , piece_id=F4S7RIS46F2C5WNN7U3PUPB2ZDHKGTPK4PSQWBY3NYRPYE75BZZQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.309133229 (566h11m1.801565673s) [noise=true , address=77.90.31.19:4842 , piece_id=SNHVHFMHNDOFRIBUFN57FMB6WW5M4F6ZSYKVKHTTIP7PHWIXFKQA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.314939261 (566h11m1.80737169s) [noise=true , address=45.91.116.13:4155 , piece_id=K36OXFO5KYKMUZ2YMXKB6BA73L27E7LMNEXKPND6VJJETJUXYHLA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.316400192 (566h11m1.808832623s) [noise=true , address=77.90.31.17:4771 , piece_id=TVY5SS3RKYSZVM3XO4M773NRJV2A2NGHJ7HDEMFR4PWVRBPBTA5A, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.319177316 (566h11m1.811609759s) [noise=true , address=77.90.31.15:4697 , piece_id=7PHDYIDCN6DH6UDFG7BRD4YFID3M3QQI6BOCZ77RNX5SFFTOPAKA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.320342285 (566h11m1.812774714s) [noise=true , address=77.90.31.21:4877 , piece_id=ILYDJR7MFKIWVVRGA43FR6RO2LZX7C5K3W23Q2WSK6ZMPZOLGTVA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.321748835 (566h11m1.814181264s) [noise=true , address=45.91.116.14:4190 , piece_id=YI3OYG6O65E5IU5XJOXXRXTTLMK2GWPTVHHHPX3MHPZC4UQL7KOA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.321987071 (566h11m1.814419516s) [noise=true , address=45.91.116.6:4973 , piece_id=OQ6SGUTPT4YDLCH42LDZHOFLIB5BXKKIE7AZKAXHHOHHABQEMDUQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:18:41.325593759 (566h11m1.818026188s) [noise=true , address=77.90.31.12:4592 , piece_id=CAL3VAEZHZRBGJVNWNVJXIWQD4WXWMTKY7PELLVJBIIPTSXJU42A, seg_pos={0 0}, upload=498096]: Done uploading piece. err: <nil> Sep 22 04:19:10.150545639 (566h11m30.642978073s) [noise=true , address=145.79.227.11:4730 , piece_id=CPTOYO4DM6CHTC4VGDQZ4AH53Z7PK7KU3VRQPKXXL53QZ75MFEQQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150606746 (566h11m30.643039178s) [noise=true , address=145.79.227.6:4259 , piece_id=H4R5MP67R3PRCWFUWOBSOMYFHYUHDSHJLQEIUEYR4EY6R52JTNJQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150655454 (566h11m30.643087895s) [noise=true , address=145.79.227.17:4972 , piece_id=ABMDOPTPGZMGXZD3S2DLBCZZHHVBLQKAPNK2KVAU7WYKJTMRYSWA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150718539 (566h11m30.643150968s) [noise=true , address=145.79.227.13:4803 , piece_id=6HR7IOH2S3KOI6FE5M4VJVXIHLEILIAFS7NVURLBM2UZC533NEYQ, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150780577 (566h11m30.643213015s) [noise=true , address=145.79.227.12:4767 , piece_id=52KI3VLSGHGJOKYPKBQLXHJZMCJ6HDVPKT7DYW2O4QISRPGTCJIA, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150825180 (566h11m30.643257627s) [noise=true , address=145.79.227.13:4801 , piece_id=Q4MHR3OPGVAUNQR5ZZNMQDJPG77EPRV4HRNBVQFUR6E2NLPLVK2A, seg_pos={0 0}, upload=498096]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:19:10.150857541 (566h11m30.643289987s) [seg_pos={0 0}, upload=498096]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:19:10.256585180 (566h11m30.749017615s) [upload=498096]: Done uploading segments... Sep 22 04:19:10.256612970 (566h11m30.749045406s) [upload=498096]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.107810797s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X.......X......X..XXXX.........................XX..X..] duration: 2.861174541s FAIL (2025-09-22 04:18:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.861196011s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.X..................X..X..XX..X......XXX.....XX.X..X.XX..X..] duration: 2.394574073s FAIL (2025-09-22 04:18:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 04:16:26.033591720 (566h8m46.526024157s) [upload=498063]: Starting upload Sep 22 04:16:26.033711028 (566h8m46.526143466s) [upload=498063]: Done starting upload Sep 22 04:16:26.033818928 (566h8m46.526251361s) [upload=498063]: Uploading segments... Sep 22 04:16:26.034181952 (566h8m46.5266144s) [upload=498063]: Got next segment. Inline: false Sep 22 04:16:26.034251325 (566h8m46.526683757s) [upload=498063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:16:26.034257308 (566h8m46.52668975s) [upload=498063]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:16:26.147532767 (566h8m46.639965201s) [seg_pos={0 0}, upload=498063]: Begin upload segment... Sep 22 04:16:26.147588865 (566h8m46.64002131s) [seg_pos={0 0}, upload=498063]: Done begin upload segment. Sep 22 04:16:26.147613278 (566h8m46.640045723s) [noise=true , address=77.90.31.21:4878 , piece_id=LPL7DPIXYICDFKJVB6H2P3MNKWNUH2URVGOPDHBTTMFVPIFCH5OA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.147625206 (566h8m46.640057648s) [noise=true , address=145.79.227.2:4115 , piece_id=XD2VVF5SA3Y7TDVHGHAETKIDCPBT6QLHSWYXR2GSXQ26BM5CXFEA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148009202 (566h8m46.640441647s) [noise=true , address=145.79.227.16:4910 , piece_id=KT7HXSY6B3VN3LBEH2H74VJFCZ7W2RCNUJXDTVL37FOGGCQMP4VA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148016223 (566h8m46.640448673s) [noise=true , address=145.79.227.3:4151 , piece_id=U35R7OQ7SOPC2I3D52KZZL4PYM7SLYGHNRDYA2S42K4TGGAPBH4A, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148325882 (566h8m46.640758333s) [noise=true , address=45.91.116.6:4974 , piece_id=WKT4IBQLXRR2ABWKGRVIQ4XYYOU6QXPH65FVWYMTRR4LSO4CJ2IA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148622195 (566h8m46.641054637s) [noise=true , address=145.79.227.19:4077 , piece_id=NVE2GAL2EYTFCNP2QMEZPPS4SQNURVATYJVLKLJWTVLDKBUGGITQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148950823 (566h8m46.641383268s) [noise=true , address=77.90.31.10:4519 , piece_id=ABQWAKSURTGDO4WGEE7W7MRK7SQIBBWXQ2AE4JXKVG27VX4YCMJA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149281322 (566h8m46.641713775s) [noise=true , address=145.79.227.7:4293 , piece_id=WZO2U7V5YOEIOXTXIITPRTWAI4VBQPYNUPPCVRKIPBR4LTOOLJ3Q, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149571542 (566h8m46.642003986s) [noise=true , address=145.79.227.15:4875 , piece_id=5ZUDC65P2BS3Q5ORJH5YCBHAGEXJEARN3T4K6A6LMYAXHQJY6TCA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149823079 (566h8m46.642255528s) [noise=true , address=45.91.116.6:4973 , piece_id=VWNJSLIQRE4ZMDWWAN363MA4SGYSA7AHHBJ3NDZROGIZMUVGSUUQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150229803 (566h8m46.642662254s) [noise=true , address=77.90.31.21:4877 , piece_id=D55BARIHKXZIZXRKYQYAHC75NX5EUTAS3PPAGK4U6VEK2WMRJ6SQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150330895 (566h8m46.64276334s) [noise=true , address=77.90.31.16:4735 , piece_id=LRJ6UA6WJC33FIGWGBREKR4BSTT6B4YACDLVZ7LEEWKJXIZSGDIA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150644287 (566h8m46.643076728s) [noise=true , address=145.79.227.15:4873 , piece_id=AKQ5GYQGDTR2ISC5WLUHL2XIHMVRMG4DP3PEZIC6DDMBTHCCOIIA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150975481 (566h8m46.643407929s) [noise=true , address=45.91.116.14:4190 , piece_id=JEWJDNGJWC4QIDGPTUDEFVRLNLYZLUF2RK5FUWS6GZ2B4QMPIKDQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.151248420 (566h8m46.643680867s) [noise=true , address=45.91.116.5:4298 , piece_id=AOMFO5XP7RXW5JE5XD25RRRG4WBCUIV4VMTL2FESOULI656G3HUQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.151625695 (566h8m46.644058143s) [noise=true , address=45.91.116.4:4940 , piece_id=AWDT5Q3RJ4Y5MUBWEINEUEMIF3PVKM643RW4UNF5ZK47S4TBAULQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.760401381 (566h8m47.252833817s) [upload=498063]: Next returned nil segment Sep 22 04:16:26.760419233 (566h8m47.252851677s) [upload=498063]: Waiting for error group managing segments... Sep 22 04:16:26.897046618 (566h8m47.389479066s) [noise=true , address=45.91.116.14:4190 , piece_id=JEWJDNGJWC4QIDGPTUDEFVRLNLYZLUF2RK5FUWS6GZ2B4QMPIKDQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.897901981 (566h8m47.390334417s) [noise=true , address=45.91.116.6:4974 , piece_id=WKT4IBQLXRR2ABWKGRVIQ4XYYOU6QXPH65FVWYMTRR4LSO4CJ2IA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.898042720 (566h8m47.390475152s) [noise=true , address=77.90.31.16:4735 , piece_id=LRJ6UA6WJC33FIGWGBREKR4BSTT6B4YACDLVZ7LEEWKJXIZSGDIA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.903353004 (566h8m47.395785432s) [noise=true , address=77.90.31.21:4877 , piece_id=D55BARIHKXZIZXRKYQYAHC75NX5EUTAS3PPAGK4U6VEK2WMRJ6SQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.909699233 (566h8m47.402131693s) [noise=true , address=45.91.116.5:4298 , piece_id=AOMFO5XP7RXW5JE5XD25RRRG4WBCUIV4VMTL2FESOULI656G3HUQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.910583149 (566h8m47.403015601s) [noise=true , address=45.91.116.4:4940 , piece_id=AWDT5Q3RJ4Y5MUBWEINEUEMIF3PVKM643RW4UNF5ZK47S4TBAULQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.910849128 (566h8m47.403281572s) [noise=true , address=45.91.116.6:4973 , piece_id=VWNJSLIQRE4ZMDWWAN363MA4SGYSA7AHHBJ3NDZROGIZMUVGSUUQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.916417428 (566h8m47.408849856s) [noise=true , address=77.90.31.10:4519 , piece_id=ABQWAKSURTGDO4WGEE7W7MRK7SQIBBWXQ2AE4JXKVG27VX4YCMJA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.918514315 (566h8m47.41094676s) [noise=true , address=77.90.31.21:4878 , piece_id=LPL7DPIXYICDFKJVB6H2P3MNKWNUH2URVGOPDHBTTMFVPIFCH5OA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:27.943991525 (566h8m48.436423978s) [noise=true , address=145.79.227.16:4910 , piece_id=KT7HXSY6B3VN3LBEH2H74VJFCZ7W2RCNUJXDTVL37FOGGCQMP4VA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735006754 (566h9m16.227439197s) [noise=true , address=145.79.227.19:4077 , piece_id=NVE2GAL2EYTFCNP2QMEZPPS4SQNURVATYJVLKLJWTVLDKBUGGITQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735081388 (566h9m16.227513822s) [noise=true , address=145.79.227.15:4873 , piece_id=AKQ5GYQGDTR2ISC5WLUHL2XIHMVRMG4DP3PEZIC6DDMBTHCCOIIA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735226700 (566h9m16.227659133s) [noise=true , address=145.79.227.3:4151 , piece_id=U35R7OQ7SOPC2I3D52KZZL4PYM7SLYGHNRDYA2S42K4TGGAPBH4A, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735430772 (566h9m16.227863208s) [noise=true , address=145.79.227.15:4875 , piece_id=5ZUDC65P2BS3Q5ORJH5YCBHAGEXJEARN3T4K6A6LMYAXHQJY6TCA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735584939 (566h9m16.228017383s) [noise=true , address=145.79.227.7:4293 , piece_id=WZO2U7V5YOEIOXTXIITPRTWAI4VBQPYNUPPCVRKIPBR4LTOOLJ3Q, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735764283 (566h9m16.22819673s) [noise=true , address=145.79.227.2:4115 , piece_id=XD2VVF5SA3Y7TDVHGHAETKIDCPBT6QLHSWYXR2GSXQ26BM5CXFEA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.736222901 (566h9m16.228655354s) [seg_pos={0 0}, upload=498063]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 04:16:55.852550244 (566h9m16.344982679s) [upload=498063]: Done uploading segments... Sep 22 04:16:55.852585912 (566h9m16.345018355s) [upload=498063]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.394677562s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X.......X......X..XXXX.........................XX..X.] duration: 2.804502528s FAIL (2025-09-22 04:17:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.804526394s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [X.X..................X..X..XX..X......XXX.....XX.X..X.XX..X.] duration: 2.46690541s FAIL (2025-09-22 04:16:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 04:16:26.033591720 (566h8m46.526024157s) [upload=498063]: Starting upload Sep 22 04:16:26.033711028 (566h8m46.526143466s) [upload=498063]: Done starting upload Sep 22 04:16:26.033818928 (566h8m46.526251361s) [upload=498063]: Uploading segments... Sep 22 04:16:26.034181952 (566h8m46.5266144s) [upload=498063]: Got next segment. Inline: false Sep 22 04:16:26.034251325 (566h8m46.526683757s) [upload=498063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:16:26.034257308 (566h8m46.52668975s) [upload=498063]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:16:26.147532767 (566h8m46.639965201s) [seg_pos={0 0}, upload=498063]: Begin upload segment... Sep 22 04:16:26.147588865 (566h8m46.64002131s) [seg_pos={0 0}, upload=498063]: Done begin upload segment. Sep 22 04:16:26.147613278 (566h8m46.640045723s) [noise=true , address=77.90.31.21:4878 , piece_id=LPL7DPIXYICDFKJVB6H2P3MNKWNUH2URVGOPDHBTTMFVPIFCH5OA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.147625206 (566h8m46.640057648s) [noise=true , address=145.79.227.2:4115 , piece_id=XD2VVF5SA3Y7TDVHGHAETKIDCPBT6QLHSWYXR2GSXQ26BM5CXFEA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148009202 (566h8m46.640441647s) [noise=true , address=145.79.227.16:4910 , piece_id=KT7HXSY6B3VN3LBEH2H74VJFCZ7W2RCNUJXDTVL37FOGGCQMP4VA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148016223 (566h8m46.640448673s) [noise=true , address=145.79.227.3:4151 , piece_id=U35R7OQ7SOPC2I3D52KZZL4PYM7SLYGHNRDYA2S42K4TGGAPBH4A, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148325882 (566h8m46.640758333s) [noise=true , address=45.91.116.6:4974 , piece_id=WKT4IBQLXRR2ABWKGRVIQ4XYYOU6QXPH65FVWYMTRR4LSO4CJ2IA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148622195 (566h8m46.641054637s) [noise=true , address=145.79.227.19:4077 , piece_id=NVE2GAL2EYTFCNP2QMEZPPS4SQNURVATYJVLKLJWTVLDKBUGGITQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148950823 (566h8m46.641383268s) [noise=true , address=77.90.31.10:4519 , piece_id=ABQWAKSURTGDO4WGEE7W7MRK7SQIBBWXQ2AE4JXKVG27VX4YCMJA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149281322 (566h8m46.641713775s) [noise=true , address=145.79.227.7:4293 , piece_id=WZO2U7V5YOEIOXTXIITPRTWAI4VBQPYNUPPCVRKIPBR4LTOOLJ3Q, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149571542 (566h8m46.642003986s) [noise=true , address=145.79.227.15:4875 , piece_id=5ZUDC65P2BS3Q5ORJH5YCBHAGEXJEARN3T4K6A6LMYAXHQJY6TCA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149823079 (566h8m46.642255528s) [noise=true , address=45.91.116.6:4973 , piece_id=VWNJSLIQRE4ZMDWWAN363MA4SGYSA7AHHBJ3NDZROGIZMUVGSUUQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150229803 (566h8m46.642662254s) [noise=true , address=77.90.31.21:4877 , piece_id=D55BARIHKXZIZXRKYQYAHC75NX5EUTAS3PPAGK4U6VEK2WMRJ6SQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150330895 (566h8m46.64276334s) [noise=true , address=77.90.31.16:4735 , piece_id=LRJ6UA6WJC33FIGWGBREKR4BSTT6B4YACDLVZ7LEEWKJXIZSGDIA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150644287 (566h8m46.643076728s) [noise=true , address=145.79.227.15:4873 , piece_id=AKQ5GYQGDTR2ISC5WLUHL2XIHMVRMG4DP3PEZIC6DDMBTHCCOIIA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150975481 (566h8m46.643407929s) [noise=true , address=45.91.116.14:4190 , piece_id=JEWJDNGJWC4QIDGPTUDEFVRLNLYZLUF2RK5FUWS6GZ2B4QMPIKDQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.151248420 (566h8m46.643680867s) [noise=true , address=45.91.116.5:4298 , piece_id=AOMFO5XP7RXW5JE5XD25RRRG4WBCUIV4VMTL2FESOULI656G3HUQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.151625695 (566h8m46.644058143s) [noise=true , address=45.91.116.4:4940 , piece_id=AWDT5Q3RJ4Y5MUBWEINEUEMIF3PVKM643RW4UNF5ZK47S4TBAULQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.760401381 (566h8m47.252833817s) [upload=498063]: Next returned nil segment Sep 22 04:16:26.760419233 (566h8m47.252851677s) [upload=498063]: Waiting for error group managing segments... Sep 22 04:16:26.897046618 (566h8m47.389479066s) [noise=true , address=45.91.116.14:4190 , piece_id=JEWJDNGJWC4QIDGPTUDEFVRLNLYZLUF2RK5FUWS6GZ2B4QMPIKDQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.897901981 (566h8m47.390334417s) [noise=true , address=45.91.116.6:4974 , piece_id=WKT4IBQLXRR2ABWKGRVIQ4XYYOU6QXPH65FVWYMTRR4LSO4CJ2IA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.898042720 (566h8m47.390475152s) [noise=true , address=77.90.31.16:4735 , piece_id=LRJ6UA6WJC33FIGWGBREKR4BSTT6B4YACDLVZ7LEEWKJXIZSGDIA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.903353004 (566h8m47.395785432s) [noise=true , address=77.90.31.21:4877 , piece_id=D55BARIHKXZIZXRKYQYAHC75NX5EUTAS3PPAGK4U6VEK2WMRJ6SQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.909699233 (566h8m47.402131693s) [noise=true , address=45.91.116.5:4298 , piece_id=AOMFO5XP7RXW5JE5XD25RRRG4WBCUIV4VMTL2FESOULI656G3HUQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.910583149 (566h8m47.403015601s) [noise=true , address=45.91.116.4:4940 , piece_id=AWDT5Q3RJ4Y5MUBWEINEUEMIF3PVKM643RW4UNF5ZK47S4TBAULQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.910849128 (566h8m47.403281572s) [noise=true , address=45.91.116.6:4973 , piece_id=VWNJSLIQRE4ZMDWWAN363MA4SGYSA7AHHBJ3NDZROGIZMUVGSUUQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.916417428 (566h8m47.408849856s) [noise=true , address=77.90.31.10:4519 , piece_id=ABQWAKSURTGDO4WGEE7W7MRK7SQIBBWXQ2AE4JXKVG27VX4YCMJA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.918514315 (566h8m47.41094676s) [noise=true , address=77.90.31.21:4878 , piece_id=LPL7DPIXYICDFKJVB6H2P3MNKWNUH2URVGOPDHBTTMFVPIFCH5OA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:27.943991525 (566h8m48.436423978s) [noise=true , address=145.79.227.16:4910 , piece_id=KT7HXSY6B3VN3LBEH2H74VJFCZ7W2RCNUJXDTVL37FOGGCQMP4VA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735006754 (566h9m16.227439197s) [noise=true , address=145.79.227.19:4077 , piece_id=NVE2GAL2EYTFCNP2QMEZPPS4SQNURVATYJVLKLJWTVLDKBUGGITQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735081388 (566h9m16.227513822s) [noise=true , address=145.79.227.15:4873 , piece_id=AKQ5GYQGDTR2ISC5WLUHL2XIHMVRMG4DP3PEZIC6DDMBTHCCOIIA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735226700 (566h9m16.227659133s) [noise=true , address=145.79.227.3:4151 , piece_id=U35R7OQ7SOPC2I3D52KZZL4PYM7SLYGHNRDYA2S42K4TGGAPBH4A, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735430772 (566h9m16.227863208s) [noise=true , address=145.79.227.15:4875 , piece_id=5ZUDC65P2BS3Q5ORJH5YCBHAGEXJEARN3T4K6A6LMYAXHQJY6TCA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735584939 (566h9m16.228017383s) [noise=true , address=145.79.227.7:4293 , piece_id=WZO2U7V5YOEIOXTXIITPRTWAI4VBQPYNUPPCVRKIPBR4LTOOLJ3Q, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735764283 (566h9m16.22819673s) [noise=true , address=145.79.227.2:4115 , piece_id=XD2VVF5SA3Y7TDVHGHAETKIDCPBT6QLHSWYXR2GSXQ26BM5CXFEA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.736222901 (566h9m16.228655354s) [seg_pos={0 0}, upload=498063]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 04:16:55.852550244 (566h9m16.344982679s) [upload=498063]: Done uploading segments... Sep 22 04:16:55.852585912 (566h9m16.345018355s) [upload=498063]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.466931032s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.X.X..................X..X..XX..X......XXX.....XX.X..X.XX..X] duration: 30.1189162s FAIL (2025-09-22 04:16:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 04:16:26.033591720 (566h8m46.526024157s) [upload=498063]: Starting upload Sep 22 04:16:26.033711028 (566h8m46.526143466s) [upload=498063]: Done starting upload Sep 22 04:16:26.033818928 (566h8m46.526251361s) [upload=498063]: Uploading segments... Sep 22 04:16:26.034181952 (566h8m46.5266144s) [upload=498063]: Got next segment. Inline: false Sep 22 04:16:26.034251325 (566h8m46.526683757s) [upload=498063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:16:26.034257308 (566h8m46.52668975s) [upload=498063]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:16:26.147532767 (566h8m46.639965201s) [seg_pos={0 0}, upload=498063]: Begin upload segment... Sep 22 04:16:26.147588865 (566h8m46.64002131s) [seg_pos={0 0}, upload=498063]: Done begin upload segment. Sep 22 04:16:26.147613278 (566h8m46.640045723s) [noise=true , address=77.90.31.21:4878 , piece_id=LPL7DPIXYICDFKJVB6H2P3MNKWNUH2URVGOPDHBTTMFVPIFCH5OA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.147625206 (566h8m46.640057648s) [noise=true , address=145.79.227.2:4115 , piece_id=XD2VVF5SA3Y7TDVHGHAETKIDCPBT6QLHSWYXR2GSXQ26BM5CXFEA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148009202 (566h8m46.640441647s) [noise=true , address=145.79.227.16:4910 , piece_id=KT7HXSY6B3VN3LBEH2H74VJFCZ7W2RCNUJXDTVL37FOGGCQMP4VA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148016223 (566h8m46.640448673s) [noise=true , address=145.79.227.3:4151 , piece_id=U35R7OQ7SOPC2I3D52KZZL4PYM7SLYGHNRDYA2S42K4TGGAPBH4A, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148325882 (566h8m46.640758333s) [noise=true , address=45.91.116.6:4974 , piece_id=WKT4IBQLXRR2ABWKGRVIQ4XYYOU6QXPH65FVWYMTRR4LSO4CJ2IA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148622195 (566h8m46.641054637s) [noise=true , address=145.79.227.19:4077 , piece_id=NVE2GAL2EYTFCNP2QMEZPPS4SQNURVATYJVLKLJWTVLDKBUGGITQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.148950823 (566h8m46.641383268s) [noise=true , address=77.90.31.10:4519 , piece_id=ABQWAKSURTGDO4WGEE7W7MRK7SQIBBWXQ2AE4JXKVG27VX4YCMJA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149281322 (566h8m46.641713775s) [noise=true , address=145.79.227.7:4293 , piece_id=WZO2U7V5YOEIOXTXIITPRTWAI4VBQPYNUPPCVRKIPBR4LTOOLJ3Q, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149571542 (566h8m46.642003986s) [noise=true , address=145.79.227.15:4875 , piece_id=5ZUDC65P2BS3Q5ORJH5YCBHAGEXJEARN3T4K6A6LMYAXHQJY6TCA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.149823079 (566h8m46.642255528s) [noise=true , address=45.91.116.6:4973 , piece_id=VWNJSLIQRE4ZMDWWAN363MA4SGYSA7AHHBJ3NDZROGIZMUVGSUUQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150229803 (566h8m46.642662254s) [noise=true , address=77.90.31.21:4877 , piece_id=D55BARIHKXZIZXRKYQYAHC75NX5EUTAS3PPAGK4U6VEK2WMRJ6SQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150330895 (566h8m46.64276334s) [noise=true , address=77.90.31.16:4735 , piece_id=LRJ6UA6WJC33FIGWGBREKR4BSTT6B4YACDLVZ7LEEWKJXIZSGDIA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150644287 (566h8m46.643076728s) [noise=true , address=145.79.227.15:4873 , piece_id=AKQ5GYQGDTR2ISC5WLUHL2XIHMVRMG4DP3PEZIC6DDMBTHCCOIIA, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.150975481 (566h8m46.643407929s) [noise=true , address=45.91.116.14:4190 , piece_id=JEWJDNGJWC4QIDGPTUDEFVRLNLYZLUF2RK5FUWS6GZ2B4QMPIKDQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.151248420 (566h8m46.643680867s) [noise=true , address=45.91.116.5:4298 , piece_id=AOMFO5XP7RXW5JE5XD25RRRG4WBCUIV4VMTL2FESOULI656G3HUQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.151625695 (566h8m46.644058143s) [noise=true , address=45.91.116.4:4940 , piece_id=AWDT5Q3RJ4Y5MUBWEINEUEMIF3PVKM643RW4UNF5ZK47S4TBAULQ, seg_pos={0 0}, upload=498063]: Uploading piece... Sep 22 04:16:26.760401381 (566h8m47.252833817s) [upload=498063]: Next returned nil segment Sep 22 04:16:26.760419233 (566h8m47.252851677s) [upload=498063]: Waiting for error group managing segments... Sep 22 04:16:26.897046618 (566h8m47.389479066s) [noise=true , address=45.91.116.14:4190 , piece_id=JEWJDNGJWC4QIDGPTUDEFVRLNLYZLUF2RK5FUWS6GZ2B4QMPIKDQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.897901981 (566h8m47.390334417s) [noise=true , address=45.91.116.6:4974 , piece_id=WKT4IBQLXRR2ABWKGRVIQ4XYYOU6QXPH65FVWYMTRR4LSO4CJ2IA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.898042720 (566h8m47.390475152s) [noise=true , address=77.90.31.16:4735 , piece_id=LRJ6UA6WJC33FIGWGBREKR4BSTT6B4YACDLVZ7LEEWKJXIZSGDIA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.903353004 (566h8m47.395785432s) [noise=true , address=77.90.31.21:4877 , piece_id=D55BARIHKXZIZXRKYQYAHC75NX5EUTAS3PPAGK4U6VEK2WMRJ6SQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.909699233 (566h8m47.402131693s) [noise=true , address=45.91.116.5:4298 , piece_id=AOMFO5XP7RXW5JE5XD25RRRG4WBCUIV4VMTL2FESOULI656G3HUQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.910583149 (566h8m47.403015601s) [noise=true , address=45.91.116.4:4940 , piece_id=AWDT5Q3RJ4Y5MUBWEINEUEMIF3PVKM643RW4UNF5ZK47S4TBAULQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.910849128 (566h8m47.403281572s) [noise=true , address=45.91.116.6:4973 , piece_id=VWNJSLIQRE4ZMDWWAN363MA4SGYSA7AHHBJ3NDZROGIZMUVGSUUQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.916417428 (566h8m47.408849856s) [noise=true , address=77.90.31.10:4519 , piece_id=ABQWAKSURTGDO4WGEE7W7MRK7SQIBBWXQ2AE4JXKVG27VX4YCMJA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:26.918514315 (566h8m47.41094676s) [noise=true , address=77.90.31.21:4878 , piece_id=LPL7DPIXYICDFKJVB6H2P3MNKWNUH2URVGOPDHBTTMFVPIFCH5OA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: <nil> Sep 22 04:16:27.943991525 (566h8m48.436423978s) [noise=true , address=145.79.227.16:4910 , piece_id=KT7HXSY6B3VN3LBEH2H74VJFCZ7W2RCNUJXDTVL37FOGGCQMP4VA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735006754 (566h9m16.227439197s) [noise=true , address=145.79.227.19:4077 , piece_id=NVE2GAL2EYTFCNP2QMEZPPS4SQNURVATYJVLKLJWTVLDKBUGGITQ, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735081388 (566h9m16.227513822s) [noise=true , address=145.79.227.15:4873 , piece_id=AKQ5GYQGDTR2ISC5WLUHL2XIHMVRMG4DP3PEZIC6DDMBTHCCOIIA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735226700 (566h9m16.227659133s) [noise=true , address=145.79.227.3:4151 , piece_id=U35R7OQ7SOPC2I3D52KZZL4PYM7SLYGHNRDYA2S42K4TGGAPBH4A, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735430772 (566h9m16.227863208s) [noise=true , address=145.79.227.15:4875 , piece_id=5ZUDC65P2BS3Q5ORJH5YCBHAGEXJEARN3T4K6A6LMYAXHQJY6TCA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735584939 (566h9m16.228017383s) [noise=true , address=145.79.227.7:4293 , piece_id=WZO2U7V5YOEIOXTXIITPRTWAI4VBQPYNUPPCVRKIPBR4LTOOLJ3Q, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.735764283 (566h9m16.22819673s) [noise=true , address=145.79.227.2:4115 , piece_id=XD2VVF5SA3Y7TDVHGHAETKIDCPBT6QLHSWYXR2GSXQ26BM5CXFEA, seg_pos={0 0}, upload=498063]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:16:55.736222901 (566h9m16.228655354s) [seg_pos={0 0}, upload=498063]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 04:16:55.852550244 (566h9m16.344982679s) [upload=498063]: Done uploading segments... Sep 22 04:16:55.852585912 (566h9m16.345018355s) [upload=498063]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.118945747s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........X.......X......X..XXXX.........................XX..X] duration: 30.001826625s FAIL (2025-09-22 04:16:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00188019s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........X.......X......X..XXXX.........................XX..] duration: 3.067925118s FAIL (2025-09-22 04:16:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.068027971s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [X.X.X..................X..X..XX..X......XXX.....XX.X..X.XX..] duration: 2.523778766s FAIL (2025-09-22 04:15:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:14:10.966461821 (566h6m31.458894264s) [upload=498027]: Starting upload Sep 22 04:14:10.966532978 (566h6m31.458965426s) [upload=498027]: Done starting upload Sep 22 04:14:10.966663731 (566h6m31.459096178s) [upload=498027]: Uploading segments... Sep 22 04:14:10.966850459 (566h6m31.45928289s) [upload=498027]: Got next segment. Inline: false Sep 22 04:14:10.966906028 (566h6m31.459338471s) [upload=498027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:14:10.966911959 (566h6m31.459344401s) [upload=498027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:14:11.088485449 (566h6m31.580917901s) [seg_pos={0 0}, upload=498027]: Begin upload segment... Sep 22 04:14:11.088566302 (566h6m31.580998754s) [seg_pos={0 0}, upload=498027]: Done begin upload segment. Sep 22 04:14:11.088625803 (566h6m31.581058245s) [noise=true , address=45.91.116.9:4261 , piece_id=J4TO7YLAJCQFHXU2UZSQAU324EDU6OSGOWPP4MUGQ2BW4KR7SWAQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089183913 (566h6m31.581616377s) [noise=true , address=145.79.227.18:4006 , piece_id=IWHLIBAFB22P6X7DVH4YFWS7IAZ4TIRDJGTKRN76BBD6YHQCHEMQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089647265 (566h6m31.582079716s) [noise=true , address=45.91.116.10:4081 , piece_id=I4CKWYVQWT5LD3NG2A4UHYDJATDCRHPYVZ7SKOHWLI6JFLCAU7QQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089808394 (566h6m31.582240836s) [noise=true , address=145.79.227.11:4731 , piece_id=OCERGHQDYL2ZP3PJBWKYG73FGCTLRGEGB5IMCX7EZOMCSHFUF4GA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.090638965 (566h6m31.583071417s) [noise=true , address=77.90.31.16:4733 , piece_id=P6UF56DE7B7W7B2Q6QUS6ZLZJYOQJVQZC35CP3JYRSS6GVOC7J2A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.090856345 (566h6m31.583288787s) [noise=true , address=145.79.227.5:4222 , piece_id=KW3K4EB7YGHON5OT6YAPAZY7ONIWNBF7H6ISKHSUHZUDTWCVPHGA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.091653368 (566h6m31.584085821s) [noise=true , address=77.90.31.15:4699 , piece_id=GYN2JHKXDIQM265UHJE27D5S3FUVYNCIN6XUHZ7VANGGEXJ6KBFA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.092545596 (566h6m31.584978047s) [noise=true , address=145.79.227.6:4257 , piece_id=DAGZWD7S6P45UYQVIPR5VTL74XPVMAOLY3WXCNJ73IRYOFQZY6AQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.092965130 (566h6m31.585397582s) [noise=true , address=45.91.116.14:4190 , piece_id=I6SO63DPFL4HVINNHABFQD2NSJLVRUAVIAKF3MBEGOW6CZJGJT5Q, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.093338296 (566h6m31.585770742s) [noise=true , address=145.79.227.17:4971 , piece_id=RRRPVY6RQSXVBBVF5S7PLU4FLVZKZ4WEI5BGFEB3HLH2I3TGJXGA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.094939644 (566h6m31.587372095s) [noise=true , address=77.90.31.15:4698 , piece_id=OKLF75WPSCADU76AII6ZJN2EKB5PFNKZ4JN2YQ5TRAXFMOKLKYCA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.095423484 (566h6m31.587855931s) [noise=true , address=145.79.227.14:4840 , piece_id=65NXN5NHAN24HQ7Q3VGK3H2FCQZXYGPVAPQLJBIQJZXM3EEIXRKA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.095799639 (566h6m31.588232083s) [noise=true , address=77.90.31.14:4661 , piece_id=CQR5KDIGKNMYNGHVUZC7JNXW5YMIX5AXW5KPWMSNW4GAKEHJ7O6A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.096153477 (566h6m31.58858592s) [noise=true , address=45.91.116.13:4155 , piece_id=6LAWZQINFGH7RP4BNKM34GUQORJHLUJOD22QBBLRVQCLZ6WGD57A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.097008318 (566h6m31.589440773s) [noise=true , address=45.91.116.5:4299 , piece_id=VXG7OQNSTWTTTIIZJYEP5PFM5OWE5Z2N3KZGE2CGS3NNL454GPUA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.098864837 (566h6m31.591297289s) [noise=true , address=77.90.31.11:4553 , piece_id=4HO4NHKHZBTPFVN7IIOYCHCS77KFWJGKRCFMK56Q4M6CEXZCYXQA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.696689766 (566h6m32.189122208s) [upload=498027]: Next returned nil segment Sep 22 04:14:11.696710328 (566h6m32.189142771s) [upload=498027]: Waiting for error group managing segments... Sep 22 04:14:11.832344786 (566h6m32.324777217s) [noise=true , address=45.91.116.10:4081 , piece_id=I4CKWYVQWT5LD3NG2A4UHYDJATDCRHPYVZ7SKOHWLI6JFLCAU7QQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.837741129 (566h6m32.330173565s) [noise=true , address=77.90.31.16:4733 , piece_id=P6UF56DE7B7W7B2Q6QUS6ZLZJYOQJVQZC35CP3JYRSS6GVOC7J2A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.845594274 (566h6m32.338026721s) [noise=true , address=77.90.31.15:4699 , piece_id=GYN2JHKXDIQM265UHJE27D5S3FUVYNCIN6XUHZ7VANGGEXJ6KBFA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.846650069 (566h6m32.339082512s) [noise=true , address=45.91.116.14:4190 , piece_id=I6SO63DPFL4HVINNHABFQD2NSJLVRUAVIAKF3MBEGOW6CZJGJT5Q, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.848040834 (566h6m32.340473266s) [noise=true , address=45.91.116.13:4155 , piece_id=6LAWZQINFGH7RP4BNKM34GUQORJHLUJOD22QBBLRVQCLZ6WGD57A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.849066489 (566h6m32.341498918s) [noise=true , address=45.91.116.5:4299 , piece_id=VXG7OQNSTWTTTIIZJYEP5PFM5OWE5Z2N3KZGE2CGS3NNL454GPUA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.853017051 (566h6m32.345449489s) [noise=true , address=77.90.31.14:4661 , piece_id=CQR5KDIGKNMYNGHVUZC7JNXW5YMIX5AXW5KPWMSNW4GAKEHJ7O6A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.854295693 (566h6m32.346728129s) [noise=true , address=77.90.31.11:4553 , piece_id=4HO4NHKHZBTPFVN7IIOYCHCS77KFWJGKRCFMK56Q4M6CEXZCYXQA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.854362405 (566h6m32.346794853s) [noise=true , address=77.90.31.15:4698 , piece_id=OKLF75WPSCADU76AII6ZJN2EKB5PFNKZ4JN2YQ5TRAXFMOKLKYCA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.882378127 (566h6m32.374810576s) [noise=true , address=45.91.116.9:4261 , piece_id=J4TO7YLAJCQFHXU2UZSQAU324EDU6OSGOWPP4MUGQ2BW4KR7SWAQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:12.549891673 (566h6m33.042324137s) [noise=true , address=145.79.227.5:4222 , piece_id=KW3K4EB7YGHON5OT6YAPAZY7ONIWNBF7H6ISKHSUHZUDTWCVPHGA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: connect: no route to host; conn closed: op Sep 22 04:14:13.833568622 (566h6m34.326001076s) [noise=true , address=145.79.227.11:4731 , piece_id=OCERGHQDYL2ZP3PJBWKYG73FGCTLRGEGB5IMCX7EZOMCSHFUF4GA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666350500 (566h7m1.158782935s) [noise=true , address=145.79.227.6:4257 , piece_id=DAGZWD7S6P45UYQVIPR5VTL74XPVMAOLY3WXCNJ73IRYOFQZY6AQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: rpc: dial tcp 145.79.227.6:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666401736 (566h7m1.158834185s) [noise=true , address=145.79.227.17:4971 , piece_id=RRRPVY6RQSXVBBVF5S7PLU4FLVZKZ4WEI5BGFEB3HLH2I3TGJXGA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666427861 (566h7m1.158860293s) [noise=true , address=145.79.227.18:4006 , piece_id=IWHLIBAFB22P6X7DVH4YFWS7IAZ4TIRDJGTKRN76BBD6YHQCHEMQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: rpc: dial tcp 145.79.227.18:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666477992 (566h7m1.158910435s) [noise=true , address=145.79.227.14:4840 , piece_id=65NXN5NHAN24HQ7Q3VGK3H2FCQZXYGPVAPQLJBIQJZXM3EEIXRKA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666514165 (566h7m1.158946618s) [seg_pos={0 0}, upload=498027]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:14:40.774326965 (566h7m1.266759405s) [upload=498027]: Done uploading segments... Sep 22 04:14:40.774376684 (566h7m1.266809136s) [upload=498027]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.523815127s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..........X.......X......X..XXXX.........................XX.] duration: 3.144032712s FAIL (2025-09-22 04:15:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.144065904s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X..........X.......X......X..XXXX.........................XX] duration: 30.001571977s FAIL (2025-09-22 04:14:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001616951s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.X.X.X..................X..X..XX..X......XXX.....XX.X..X.XX.] duration: 2.622632144s FAIL (2025-09-22 04:14:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:14:10.966461821 (566h6m31.458894264s) [upload=498027]: Starting upload Sep 22 04:14:10.966532978 (566h6m31.458965426s) [upload=498027]: Done starting upload Sep 22 04:14:10.966663731 (566h6m31.459096178s) [upload=498027]: Uploading segments... Sep 22 04:14:10.966850459 (566h6m31.45928289s) [upload=498027]: Got next segment. Inline: false Sep 22 04:14:10.966906028 (566h6m31.459338471s) [upload=498027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:14:10.966911959 (566h6m31.459344401s) [upload=498027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:14:11.088485449 (566h6m31.580917901s) [seg_pos={0 0}, upload=498027]: Begin upload segment... Sep 22 04:14:11.088566302 (566h6m31.580998754s) [seg_pos={0 0}, upload=498027]: Done begin upload segment. Sep 22 04:14:11.088625803 (566h6m31.581058245s) [noise=true , address=45.91.116.9:4261 , piece_id=J4TO7YLAJCQFHXU2UZSQAU324EDU6OSGOWPP4MUGQ2BW4KR7SWAQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089183913 (566h6m31.581616377s) [noise=true , address=145.79.227.18:4006 , piece_id=IWHLIBAFB22P6X7DVH4YFWS7IAZ4TIRDJGTKRN76BBD6YHQCHEMQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089647265 (566h6m31.582079716s) [noise=true , address=45.91.116.10:4081 , piece_id=I4CKWYVQWT5LD3NG2A4UHYDJATDCRHPYVZ7SKOHWLI6JFLCAU7QQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089808394 (566h6m31.582240836s) [noise=true , address=145.79.227.11:4731 , piece_id=OCERGHQDYL2ZP3PJBWKYG73FGCTLRGEGB5IMCX7EZOMCSHFUF4GA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.090638965 (566h6m31.583071417s) [noise=true , address=77.90.31.16:4733 , piece_id=P6UF56DE7B7W7B2Q6QUS6ZLZJYOQJVQZC35CP3JYRSS6GVOC7J2A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.090856345 (566h6m31.583288787s) [noise=true , address=145.79.227.5:4222 , piece_id=KW3K4EB7YGHON5OT6YAPAZY7ONIWNBF7H6ISKHSUHZUDTWCVPHGA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.091653368 (566h6m31.584085821s) [noise=true , address=77.90.31.15:4699 , piece_id=GYN2JHKXDIQM265UHJE27D5S3FUVYNCIN6XUHZ7VANGGEXJ6KBFA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.092545596 (566h6m31.584978047s) [noise=true , address=145.79.227.6:4257 , piece_id=DAGZWD7S6P45UYQVIPR5VTL74XPVMAOLY3WXCNJ73IRYOFQZY6AQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.092965130 (566h6m31.585397582s) [noise=true , address=45.91.116.14:4190 , piece_id=I6SO63DPFL4HVINNHABFQD2NSJLVRUAVIAKF3MBEGOW6CZJGJT5Q, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.093338296 (566h6m31.585770742s) [noise=true , address=145.79.227.17:4971 , piece_id=RRRPVY6RQSXVBBVF5S7PLU4FLVZKZ4WEI5BGFEB3HLH2I3TGJXGA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.094939644 (566h6m31.587372095s) [noise=true , address=77.90.31.15:4698 , piece_id=OKLF75WPSCADU76AII6ZJN2EKB5PFNKZ4JN2YQ5TRAXFMOKLKYCA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.095423484 (566h6m31.587855931s) [noise=true , address=145.79.227.14:4840 , piece_id=65NXN5NHAN24HQ7Q3VGK3H2FCQZXYGPVAPQLJBIQJZXM3EEIXRKA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.095799639 (566h6m31.588232083s) [noise=true , address=77.90.31.14:4661 , piece_id=CQR5KDIGKNMYNGHVUZC7JNXW5YMIX5AXW5KPWMSNW4GAKEHJ7O6A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.096153477 (566h6m31.58858592s) [noise=true , address=45.91.116.13:4155 , piece_id=6LAWZQINFGH7RP4BNKM34GUQORJHLUJOD22QBBLRVQCLZ6WGD57A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.097008318 (566h6m31.589440773s) [noise=true , address=45.91.116.5:4299 , piece_id=VXG7OQNSTWTTTIIZJYEP5PFM5OWE5Z2N3KZGE2CGS3NNL454GPUA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.098864837 (566h6m31.591297289s) [noise=true , address=77.90.31.11:4553 , piece_id=4HO4NHKHZBTPFVN7IIOYCHCS77KFWJGKRCFMK56Q4M6CEXZCYXQA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.696689766 (566h6m32.189122208s) [upload=498027]: Next returned nil segment Sep 22 04:14:11.696710328 (566h6m32.189142771s) [upload=498027]: Waiting for error group managing segments... Sep 22 04:14:11.832344786 (566h6m32.324777217s) [noise=true , address=45.91.116.10:4081 , piece_id=I4CKWYVQWT5LD3NG2A4UHYDJATDCRHPYVZ7SKOHWLI6JFLCAU7QQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.837741129 (566h6m32.330173565s) [noise=true , address=77.90.31.16:4733 , piece_id=P6UF56DE7B7W7B2Q6QUS6ZLZJYOQJVQZC35CP3JYRSS6GVOC7J2A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.845594274 (566h6m32.338026721s) [noise=true , address=77.90.31.15:4699 , piece_id=GYN2JHKXDIQM265UHJE27D5S3FUVYNCIN6XUHZ7VANGGEXJ6KBFA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.846650069 (566h6m32.339082512s) [noise=true , address=45.91.116.14:4190 , piece_id=I6SO63DPFL4HVINNHABFQD2NSJLVRUAVIAKF3MBEGOW6CZJGJT5Q, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.848040834 (566h6m32.340473266s) [noise=true , address=45.91.116.13:4155 , piece_id=6LAWZQINFGH7RP4BNKM34GUQORJHLUJOD22QBBLRVQCLZ6WGD57A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.849066489 (566h6m32.341498918s) [noise=true , address=45.91.116.5:4299 , piece_id=VXG7OQNSTWTTTIIZJYEP5PFM5OWE5Z2N3KZGE2CGS3NNL454GPUA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.853017051 (566h6m32.345449489s) [noise=true , address=77.90.31.14:4661 , piece_id=CQR5KDIGKNMYNGHVUZC7JNXW5YMIX5AXW5KPWMSNW4GAKEHJ7O6A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.854295693 (566h6m32.346728129s) [noise=true , address=77.90.31.11:4553 , piece_id=4HO4NHKHZBTPFVN7IIOYCHCS77KFWJGKRCFMK56Q4M6CEXZCYXQA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.854362405 (566h6m32.346794853s) [noise=true , address=77.90.31.15:4698 , piece_id=OKLF75WPSCADU76AII6ZJN2EKB5PFNKZ4JN2YQ5TRAXFMOKLKYCA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.882378127 (566h6m32.374810576s) [noise=true , address=45.91.116.9:4261 , piece_id=J4TO7YLAJCQFHXU2UZSQAU324EDU6OSGOWPP4MUGQ2BW4KR7SWAQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:12.549891673 (566h6m33.042324137s) [noise=true , address=145.79.227.5:4222 , piece_id=KW3K4EB7YGHON5OT6YAPAZY7ONIWNBF7H6ISKHSUHZUDTWCVPHGA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: connect: no route to host; conn closed: op Sep 22 04:14:13.833568622 (566h6m34.326001076s) [noise=true , address=145.79.227.11:4731 , piece_id=OCERGHQDYL2ZP3PJBWKYG73FGCTLRGEGB5IMCX7EZOMCSHFUF4GA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666350500 (566h7m1.158782935s) [noise=true , address=145.79.227.6:4257 , piece_id=DAGZWD7S6P45UYQVIPR5VTL74XPVMAOLY3WXCNJ73IRYOFQZY6AQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: rpc: dial tcp 145.79.227.6:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666401736 (566h7m1.158834185s) [noise=true , address=145.79.227.17:4971 , piece_id=RRRPVY6RQSXVBBVF5S7PLU4FLVZKZ4WEI5BGFEB3HLH2I3TGJXGA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666427861 (566h7m1.158860293s) [noise=true , address=145.79.227.18:4006 , piece_id=IWHLIBAFB22P6X7DVH4YFWS7IAZ4TIRDJGTKRN76BBD6YHQCHEMQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: rpc: dial tcp 145.79.227.18:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666477992 (566h7m1.158910435s) [noise=true , address=145.79.227.14:4840 , piece_id=65NXN5NHAN24HQ7Q3VGK3H2FCQZXYGPVAPQLJBIQJZXM3EEIXRKA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666514165 (566h7m1.158946618s) [seg_pos={0 0}, upload=498027]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:14:40.774326965 (566h7m1.266759405s) [upload=498027]: Done uploading segments... Sep 22 04:14:40.774376684 (566h7m1.266809136s) [upload=498027]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.622731165s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [..X.X.X..................X..X..XX..X......XXX.....XX.X..X.XX] duration: 30.110969334s FAIL (2025-09-22 04:14:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:14:10.966461821 (566h6m31.458894264s) [upload=498027]: Starting upload Sep 22 04:14:10.966532978 (566h6m31.458965426s) [upload=498027]: Done starting upload Sep 22 04:14:10.966663731 (566h6m31.459096178s) [upload=498027]: Uploading segments... Sep 22 04:14:10.966850459 (566h6m31.45928289s) [upload=498027]: Got next segment. Inline: false Sep 22 04:14:10.966906028 (566h6m31.459338471s) [upload=498027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:14:10.966911959 (566h6m31.459344401s) [upload=498027]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:14:11.088485449 (566h6m31.580917901s) [seg_pos={0 0}, upload=498027]: Begin upload segment... Sep 22 04:14:11.088566302 (566h6m31.580998754s) [seg_pos={0 0}, upload=498027]: Done begin upload segment. Sep 22 04:14:11.088625803 (566h6m31.581058245s) [noise=true , address=45.91.116.9:4261 , piece_id=J4TO7YLAJCQFHXU2UZSQAU324EDU6OSGOWPP4MUGQ2BW4KR7SWAQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089183913 (566h6m31.581616377s) [noise=true , address=145.79.227.18:4006 , piece_id=IWHLIBAFB22P6X7DVH4YFWS7IAZ4TIRDJGTKRN76BBD6YHQCHEMQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089647265 (566h6m31.582079716s) [noise=true , address=45.91.116.10:4081 , piece_id=I4CKWYVQWT5LD3NG2A4UHYDJATDCRHPYVZ7SKOHWLI6JFLCAU7QQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.089808394 (566h6m31.582240836s) [noise=true , address=145.79.227.11:4731 , piece_id=OCERGHQDYL2ZP3PJBWKYG73FGCTLRGEGB5IMCX7EZOMCSHFUF4GA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.090638965 (566h6m31.583071417s) [noise=true , address=77.90.31.16:4733 , piece_id=P6UF56DE7B7W7B2Q6QUS6ZLZJYOQJVQZC35CP3JYRSS6GVOC7J2A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.090856345 (566h6m31.583288787s) [noise=true , address=145.79.227.5:4222 , piece_id=KW3K4EB7YGHON5OT6YAPAZY7ONIWNBF7H6ISKHSUHZUDTWCVPHGA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.091653368 (566h6m31.584085821s) [noise=true , address=77.90.31.15:4699 , piece_id=GYN2JHKXDIQM265UHJE27D5S3FUVYNCIN6XUHZ7VANGGEXJ6KBFA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.092545596 (566h6m31.584978047s) [noise=true , address=145.79.227.6:4257 , piece_id=DAGZWD7S6P45UYQVIPR5VTL74XPVMAOLY3WXCNJ73IRYOFQZY6AQ, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.092965130 (566h6m31.585397582s) [noise=true , address=45.91.116.14:4190 , piece_id=I6SO63DPFL4HVINNHABFQD2NSJLVRUAVIAKF3MBEGOW6CZJGJT5Q, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.093338296 (566h6m31.585770742s) [noise=true , address=145.79.227.17:4971 , piece_id=RRRPVY6RQSXVBBVF5S7PLU4FLVZKZ4WEI5BGFEB3HLH2I3TGJXGA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.094939644 (566h6m31.587372095s) [noise=true , address=77.90.31.15:4698 , piece_id=OKLF75WPSCADU76AII6ZJN2EKB5PFNKZ4JN2YQ5TRAXFMOKLKYCA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.095423484 (566h6m31.587855931s) [noise=true , address=145.79.227.14:4840 , piece_id=65NXN5NHAN24HQ7Q3VGK3H2FCQZXYGPVAPQLJBIQJZXM3EEIXRKA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.095799639 (566h6m31.588232083s) [noise=true , address=77.90.31.14:4661 , piece_id=CQR5KDIGKNMYNGHVUZC7JNXW5YMIX5AXW5KPWMSNW4GAKEHJ7O6A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.096153477 (566h6m31.58858592s) [noise=true , address=45.91.116.13:4155 , piece_id=6LAWZQINFGH7RP4BNKM34GUQORJHLUJOD22QBBLRVQCLZ6WGD57A, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.097008318 (566h6m31.589440773s) [noise=true , address=45.91.116.5:4299 , piece_id=VXG7OQNSTWTTTIIZJYEP5PFM5OWE5Z2N3KZGE2CGS3NNL454GPUA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.098864837 (566h6m31.591297289s) [noise=true , address=77.90.31.11:4553 , piece_id=4HO4NHKHZBTPFVN7IIOYCHCS77KFWJGKRCFMK56Q4M6CEXZCYXQA, seg_pos={0 0}, upload=498027]: Uploading piece... Sep 22 04:14:11.696689766 (566h6m32.189122208s) [upload=498027]: Next returned nil segment Sep 22 04:14:11.696710328 (566h6m32.189142771s) [upload=498027]: Waiting for error group managing segments... Sep 22 04:14:11.832344786 (566h6m32.324777217s) [noise=true , address=45.91.116.10:4081 , piece_id=I4CKWYVQWT5LD3NG2A4UHYDJATDCRHPYVZ7SKOHWLI6JFLCAU7QQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.837741129 (566h6m32.330173565s) [noise=true , address=77.90.31.16:4733 , piece_id=P6UF56DE7B7W7B2Q6QUS6ZLZJYOQJVQZC35CP3JYRSS6GVOC7J2A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.845594274 (566h6m32.338026721s) [noise=true , address=77.90.31.15:4699 , piece_id=GYN2JHKXDIQM265UHJE27D5S3FUVYNCIN6XUHZ7VANGGEXJ6KBFA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.846650069 (566h6m32.339082512s) [noise=true , address=45.91.116.14:4190 , piece_id=I6SO63DPFL4HVINNHABFQD2NSJLVRUAVIAKF3MBEGOW6CZJGJT5Q, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.848040834 (566h6m32.340473266s) [noise=true , address=45.91.116.13:4155 , piece_id=6LAWZQINFGH7RP4BNKM34GUQORJHLUJOD22QBBLRVQCLZ6WGD57A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.849066489 (566h6m32.341498918s) [noise=true , address=45.91.116.5:4299 , piece_id=VXG7OQNSTWTTTIIZJYEP5PFM5OWE5Z2N3KZGE2CGS3NNL454GPUA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.853017051 (566h6m32.345449489s) [noise=true , address=77.90.31.14:4661 , piece_id=CQR5KDIGKNMYNGHVUZC7JNXW5YMIX5AXW5KPWMSNW4GAKEHJ7O6A, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.854295693 (566h6m32.346728129s) [noise=true , address=77.90.31.11:4553 , piece_id=4HO4NHKHZBTPFVN7IIOYCHCS77KFWJGKRCFMK56Q4M6CEXZCYXQA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.854362405 (566h6m32.346794853s) [noise=true , address=77.90.31.15:4698 , piece_id=OKLF75WPSCADU76AII6ZJN2EKB5PFNKZ4JN2YQ5TRAXFMOKLKYCA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:11.882378127 (566h6m32.374810576s) [noise=true , address=45.91.116.9:4261 , piece_id=J4TO7YLAJCQFHXU2UZSQAU324EDU6OSGOWPP4MUGQ2BW4KR7SWAQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: <nil> Sep 22 04:14:12.549891673 (566h6m33.042324137s) [noise=true , address=145.79.227.5:4222 , piece_id=KW3K4EB7YGHON5OT6YAPAZY7ONIWNBF7H6ISKHSUHZUDTWCVPHGA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: connect: no route to host; conn closed: op Sep 22 04:14:13.833568622 (566h6m34.326001076s) [noise=true , address=145.79.227.11:4731 , piece_id=OCERGHQDYL2ZP3PJBWKYG73FGCTLRGEGB5IMCX7EZOMCSHFUF4GA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666350500 (566h7m1.158782935s) [noise=true , address=145.79.227.6:4257 , piece_id=DAGZWD7S6P45UYQVIPR5VTL74XPVMAOLY3WXCNJ73IRYOFQZY6AQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: rpc: dial tcp 145.79.227.6:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666401736 (566h7m1.158834185s) [noise=true , address=145.79.227.17:4971 , piece_id=RRRPVY6RQSXVBBVF5S7PLU4FLVZKZ4WEI5BGFEB3HLH2I3TGJXGA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666427861 (566h7m1.158860293s) [noise=true , address=145.79.227.18:4006 , piece_id=IWHLIBAFB22P6X7DVH4YFWS7IAZ4TIRDJGTKRN76BBD6YHQCHEMQ, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: rpc: dial tcp 145.79.227.18:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666477992 (566h7m1.158910435s) [noise=true , address=145.79.227.14:4840 , piece_id=65NXN5NHAN24HQ7Q3VGK3H2FCQZXYGPVAPQLJBIQJZXM3EEIXRKA, seg_pos={0 0}, upload=498027]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:40.666514165 (566h7m1.158946618s) [seg_pos={0 0}, upload=498027]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:14:40.774326965 (566h7m1.266759405s) [upload=498027]: Done uploading segments... Sep 22 04:14:40.774376684 (566h7m1.266809136s) [upload=498027]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.110993252s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [...X.X.X..................X..X..XX..X......XXX.....XX.X..X.X] duration: 30.113982s FAIL (2025-09-22 04:14:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:13:40.656915117 (566h6m1.149347552s) [upload=498019]: Starting upload Sep 22 04:13:40.657024180 (566h6m1.149456632s) [upload=498019]: Done starting upload Sep 22 04:13:40.657168086 (566h6m1.14960053s) [upload=498019]: Uploading segments... Sep 22 04:13:40.657409664 (566h6m1.149842112s) [upload=498019]: Got next segment. Inline: false Sep 22 04:13:40.657477832 (566h6m1.149910278s) [upload=498019]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:13:40.657483833 (566h6m1.149916277s) [upload=498019]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:13:40.796093120 (566h6m1.288525553s) [seg_pos={0 0}, upload=498019]: Begin upload segment... Sep 22 04:13:40.796147443 (566h6m1.288579877s) [seg_pos={0 0}, upload=498019]: Done begin upload segment. Sep 22 04:13:40.796174795 (566h6m1.28860724s) [noise=true , address=77.90.31.16:4734 , piece_id=FNCJJERMZFORXDATDKTU4GJD5AYI5S7DTHDHLLMPQZPPKCAHGJEQ, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.796886535 (566h6m1.28931905s) [noise=true , address=145.79.227.14:4839 , piece_id=X3ATEUNXTTHQZJ3ETU6EENPTW6TAK6GXDFJDIW4QJUO2SGDTOPBQ, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.797049741 (566h6m1.289482189s) [noise=true , address=45.91.116.9:4261 , piece_id=5JXXGWHE55ZDEW3YKOW76GGAZYHWVFUF3YSJUSVCE3HS45NCCFYA, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.797420403 (566h6m1.289852853s) [noise=true , address=77.90.31.21:4879 , piece_id=PDTD463DNJH3XUQBZGEWAYUC3TMKJ3AIM4NBMM5RHXHOCHWXYHWQ, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.797431659 (566h6m1.289864113s) [noise=true , address=77.90.31.10:4517 , piece_id=5MKURZ2JWONAGOXFXDWA73EX4TN53KGSGIVJPFZVONDSJFSTROWQ, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.797776394 (566h6m1.29020885s) [noise=true , address=77.90.31.14:4661 , piece_id=O5D3FXKZH56EDEH3PFEI2ACZ3GMLBJMKIVLO3GA3FNGHVEEMXCJQ, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.797815134 (566h6m1.290247582s) [noise=true , address=45.91.116.4:4939 , piece_id=BGBVMV7B5GJXHNFPYTWRDWZOYVMZID3AWNPA72JJOUPAQ3SUKWCA, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.798146696 (566h6m1.290579146s) [noise=true , address=77.90.31.14:4662 , piece_id=ZT4XTFSOMAHH3INHEYNYXDZU2OG6WS2DINMHD3WDUOLWXW5JUS4A, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.798254304 (566h6m1.290686754s) [noise=true , address=77.90.31.19:4843 , piece_id=KJ4H6DLAU3DGU6I6Y7CA5WEXC2E5QYJSSDGOVJZIEQTJWJMLD7WA, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.798536008 (566h6m1.290968465s) [noise=true , address=145.79.227.4:4187 , piece_id=37GM5BMD3ELBBOGBE2FAW2ZZCWDNILL2QIMGUXRZ6HJMIPN7DEAQ, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.798670162 (566h6m1.291102613s) [noise=true , address=77.90.31.17:4772 , piece_id=4FOIELEEKWSB7VN7ZDDJD3YS5FAFRX7BHXFDW7TXNQ46V4PKL2CQ, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.798923232 (566h6m1.291355685s) [noise=true , address=45.91.116.10:4081 , piece_id=QBJ7E55IVHHYCPGUSZWHOBKQRWPQO4F7A3MLCV6GA4DYNKJ3A2LA, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.799057861 (566h6m1.291490308s) [noise=true , address=145.79.227.2:4115 , piece_id=ZQR6ZSMUCQXYMGPF6SQGGPE46FSKPCNCFPICVR2444IVVNHE7D3Q, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.799439959 (566h6m1.291872405s) [noise=true , address=145.79.227.5:4222 , piece_id=DPT62FMMW62PANWYDS4CHD5MFSXCAKRLQYSCKT4XHWOFEXE7VEWA, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.799822023 (566h6m1.292254472s) [noise=true , address=45.91.116.14:4189 , piece_id=JKYDBWPN26XVV4XARBZYTP4MLFC7T73Q6LZNQT25MJ4XTYIDUCWA, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:40.800313227 (566h6m1.29274568s) [noise=true , address=77.90.31.10:4518 , piece_id=PPXFMPOCO53PSCKA2VSUU3BYU742MJ2LILEQ6ZDYLRVXUMEPWPNA, seg_pos={0 0}, upload=498019]: Uploading piece... Sep 22 04:13:41.417733426 (566h6m1.910165858s) [upload=498019]: Next returned nil segment Sep 22 04:13:41.417749913 (566h6m1.910182359s) [upload=498019]: Waiting for error group managing segments... Sep 22 04:13:41.552065345 (566h6m2.044497775s) [noise=true , address=45.91.116.9:4261 , piece_id=5JXXGWHE55ZDEW3YKOW76GGAZYHWVFUF3YSJUSVCE3HS45NCCFYA, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.552734519 (566h6m2.045166957s) [noise=true , address=77.90.31.14:4662 , piece_id=ZT4XTFSOMAHH3INHEYNYXDZU2OG6WS2DINMHD3WDUOLWXW5JUS4A, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.554816110 (566h6m2.047248553s) [noise=true , address=77.90.31.10:4518 , piece_id=PPXFMPOCO53PSCKA2VSUU3BYU742MJ2LILEQ6ZDYLRVXUMEPWPNA, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.555451634 (566h6m2.047884078s) [noise=true , address=45.91.116.14:4189 , piece_id=JKYDBWPN26XVV4XARBZYTP4MLFC7T73Q6LZNQT25MJ4XTYIDUCWA, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.557805255 (566h6m2.050237698s) [noise=true , address=77.90.31.19:4843 , piece_id=KJ4H6DLAU3DGU6I6Y7CA5WEXC2E5QYJSSDGOVJZIEQTJWJMLD7WA, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.560964111 (566h6m2.05339654s) [noise=true , address=77.90.31.21:4879 , piece_id=PDTD463DNJH3XUQBZGEWAYUC3TMKJ3AIM4NBMM5RHXHOCHWXYHWQ, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.561555757 (566h6m2.053988203s) [noise=true , address=45.91.116.4:4939 , piece_id=BGBVMV7B5GJXHNFPYTWRDWZOYVMZID3AWNPA72JJOUPAQ3SUKWCA, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.562209057 (566h6m2.054641514s) [noise=true , address=45.91.116.10:4081 , piece_id=QBJ7E55IVHHYCPGUSZWHOBKQRWPQO4F7A3MLCV6GA4DYNKJ3A2LA, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.567194072 (566h6m2.059626502s) [noise=true , address=77.90.31.17:4772 , piece_id=4FOIELEEKWSB7VN7ZDDJD3YS5FAFRX7BHXFDW7TXNQ46V4PKL2CQ, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.568059512 (566h6m2.060491959s) [noise=true , address=77.90.31.16:4734 , piece_id=FNCJJERMZFORXDATDKTU4GJD5AYI5S7DTHDHLLMPQZPPKCAHGJEQ, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.571301264 (566h6m2.063733708s) [noise=true , address=77.90.31.10:4517 , piece_id=5MKURZ2JWONAGOXFXDWA73EX4TN53KGSGIVJPFZVONDSJFSTROWQ, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:13:41.575035543 (566h6m2.06746799s) [noise=true , address=77.90.31.14:4661 , piece_id=O5D3FXKZH56EDEH3PFEI2ACZ3GMLBJMKIVLO3GA3FNGHVEEMXCJQ, seg_pos={0 0}, upload=498019]: Done uploading piece. err: <nil> Sep 22 04:14:10.370370356 (566h6m30.862802808s) [noise=true , address=145.79.227.4:4187 , piece_id=37GM5BMD3ELBBOGBE2FAW2ZZCWDNILL2QIMGUXRZ6HJMIPN7DEAQ, seg_pos={0 0}, upload=498019]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:10.370656939 (566h6m30.863089391s) [noise=true , address=145.79.227.14:4839 , piece_id=X3ATEUNXTTHQZJ3ETU6EENPTW6TAK6GXDFJDIW4QJUO2SGDTOPBQ, seg_pos={0 0}, upload=498019]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:10.370725683 (566h6m30.863158115s) [noise=true , address=145.79.227.2:4115 , piece_id=ZQR6ZSMUCQXYMGPF6SQGGPE46FSKPCNCFPICVR2444IVVNHE7D3Q, seg_pos={0 0}, upload=498019]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:10.370736195 (566h6m30.863168634s) [noise=true , address=145.79.227.5:4222 , piece_id=DPT62FMMW62PANWYDS4CHD5MFSXCAKRLQYSCKT4XHWOFEXE7VEWA, seg_pos={0 0}, upload=498019]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:14:10.370799116 (566h6m30.863231572s) [seg_pos={0 0}, upload=498019]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:14:10.481335489 (566h6m30.973767932s) [upload=498019]: Done uploading segments... Sep 22 04:14:10.481414342 (566h6m30.97384678s) [upload=498019]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.114014004s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XX..........X.......X......X..XXXX.........................X] duration: 30.001958712s FAIL (2025-09-22 04:14:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001992957s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [....X.X.X..................X..X..XX..X......XXX.....XX.X..X.] duration: 2.461159037s FAIL (2025-09-22 04:13:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== starting upload Sep 22 04:12:48.706801370 (566h5m9.199233813s) [upload=498007]: Uploading segments... Sep 22 04:12:48.707229293 (566h5m9.199661737s) [upload=498007]: Got next segment. Inline: false Sep 22 04:12:48.707297923 (566h5m9.199730364s) [upload=498007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:12:48.707306317 (566h5m9.199738764s) [upload=498007]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:12:48.892901897 (566h5m9.38533434s) [seg_pos={0 0}, upload=498007]: Begin upload segment... Sep 22 04:12:48.892949939 (566h5m9.385382371s) [seg_pos={0 0}, upload=498007]: Done begin upload segment. Sep 22 04:12:48.892975277 (566h5m9.38540772s) [noise=true , address=145.79.227.16:4909 , piece_id=U7LWV57HSE5QKQ7UU4I2AD7NX2B2ISJZUM5AUSK3OGBPA4ICGQ7A, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.892996534 (566h5m9.385428985s) [noise=true , address=145.79.227.3:4151 , piece_id=MRIW4AC56ZE4HZU3C7UPNZKLOFAQVK4W42K6BSEAVW3TBWP3M5BA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.893370435 (566h5m9.385802868s) [noise=true , address=145.79.227.5:4223 , piece_id=ABJ62GABVLVJJROKLPAK52PUFQJL5ECLOEEFNKGBFHLV4WFESP5A, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.894484015 (566h5m9.386916463s) [noise=true , address=145.79.227.5:4221 , piece_id=E4BKEVYABCE7XNYHGVL327IMJ4NVVSDVJKMO2KZSC5I2SSDNAWQQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.895012848 (566h5m9.387445305s) [noise=true , address=45.91.116.11:4117 , piece_id=J3EJ4A564X6SKQY74VLGP7LOGUJQANLHDU2PZR5KZ4N5CDYIGTVQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.896223753 (566h5m9.388656205s) [noise=true , address=145.79.227.4:4187 , piece_id=D6EIKDGF6ZWM76AACFCVQTSPKOKHBRKUIAMFTHQI6ZMGK4FXKOUA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.897278701 (566h5m9.389711153s) [noise=true , address=45.91.116.7:4010 , piece_id=VLKLHFG5UFNGAQCF7T6XS6AQ6EH7FR32BCUVJG4IWZBLGZVTWGYQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.898535819 (566h5m9.390968272s) [noise=true , address=145.79.227.13:4804 , piece_id=FCXCLNRFJN7WXVGMVR5KIXCV5CBYYJK42BZM466QEJ4KRGEIBE5A, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.899251154 (566h5m9.391683606s) [noise=true , address=145.79.227.11:4731 , piece_id=UINF5M2UA5JF5L25X3KGFDLBHQAHTDYAWWVSS67ZPZU4YPDUPJAQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.900606817 (566h5m9.393039269s) [noise=true , address=45.91.116.4:4939 , piece_id=SOAXVODPU4IAOCAUAFRERMBDOV6KX2QWR6VHBIN45HZF522X36NA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.901103197 (566h5m9.393535649s) [noise=true , address=45.91.116.9:4263 , piece_id=6NMFEKPW3VBWILRUE2TSSQIKKRZMK6QE2UB3IVG6B6ZZTLJBOIIQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.901505931 (566h5m9.393938382s) [noise=true , address=45.91.116.5:4297 , piece_id=Q3I55TEQKGYKFOZXTFAHFOCUD6UID7CTFQPBL444XKQKSNFFKTIQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.901953049 (566h5m9.394385501s) [noise=true , address=145.79.227.7:4295 , piece_id=4HX2RRSRWSFJABQNL4EPSWLSYT24YVSAROWLJ4AKJEUM76VXLJZA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.902444972 (566h5m9.394877424s) [noise=true , address=45.91.116.10:4081 , piece_id=IRDNUJENCOCKRS2TYGOO5ZTLMBQACK2ACMEZPZCHK24PTDLPYLVQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.903683159 (566h5m9.39611561s) [noise=true , address=77.90.31.15:4697 , piece_id=K7Q5ZEIO4CHNY447GTNJTWZA67WFQEEHYUHHY6N43ATQZAOPJQ7A, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.904162769 (566h5m9.396595222s) [noise=true , address=145.79.227.19:4079 , piece_id=7ZQS2H3PLGW225NRTL2HKWEOG5KJQ6YE3FKKROBZSTCZT53VCXRA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:49.516244091 (566h5m10.008676525s) [upload=498007]: Next returned nil segment Sep 22 04:12:49.516265465 (566h5m10.008697907s) [upload=498007]: Waiting for error group managing segments... Sep 22 04:12:49.654197031 (566h5m10.14662948s) [noise=true , address=45.91.116.4:4939 , piece_id=SOAXVODPU4IAOCAUAFRERMBDOV6KX2QWR6VHBIN45HZF522X36NA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.658411093 (566h5m10.15084353s) [noise=true , address=45.91.116.10:4081 , piece_id=IRDNUJENCOCKRS2TYGOO5ZTLMBQACK2ACMEZPZCHK24PTDLPYLVQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.658650347 (566h5m10.15108279s) [noise=true , address=77.90.31.15:4697 , piece_id=K7Q5ZEIO4CHNY447GTNJTWZA67WFQEEHYUHHY6N43ATQZAOPJQ7A, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.659224641 (566h5m10.151657075s) [noise=true , address=45.91.116.7:4010 , piece_id=VLKLHFG5UFNGAQCF7T6XS6AQ6EH7FR32BCUVJG4IWZBLGZVTWGYQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.660546209 (566h5m10.152978647s) [noise=true , address=45.91.116.5:4297 , piece_id=Q3I55TEQKGYKFOZXTFAHFOCUD6UID7CTFQPBL444XKQKSNFFKTIQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.667064175 (566h5m10.159496609s) [noise=true , address=45.91.116.9:4263 , piece_id=6NMFEKPW3VBWILRUE2TSSQIKKRZMK6QE2UB3IVG6B6ZZTLJBOIIQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.668979943 (566h5m10.161412378s) [noise=true , address=45.91.116.11:4117 , piece_id=J3EJ4A564X6SKQY74VLGP7LOGUJQANLHDU2PZR5KZ4N5CDYIGTVQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:50.852999799 (566h5m11.345432232s) [noise=true , address=145.79.227.11:4731 , piece_id=UINF5M2UA5JF5L25X3KGFDLBHQAHTDYAWWVSS67ZPZU4YPDUPJAQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:04.965629969 (566h5m25.458062418s) [noise=true , address=145.79.227.5:4223 , piece_id=ABJ62GABVLVJJROKLPAK52PUFQJL5ECLOEEFNKGBFHLV4WFESP5A, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:04.965895803 (566h5m25.458328236s) [noise=true , address=145.79.227.5:4221 , piece_id=E4BKEVYABCE7XNYHGVL327IMJ4NVVSDVJKMO2KZSC5I2SSDNAWQQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: rpc: dial tcp 145.79.227.5:4221: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:05.000653869 (566h5m25.49308636s) [noise=true , address=145.79.227.16:4909 , piece_id=U7LWV57HSE5QKQ7UU4I2AD7NX2B2ISJZUM5AUSK3OGBPA4ICGQ7A, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 04:13:18.399424194 (566h5m38.891856637s) [noise=true , address=145.79.227.4:4187 , piece_id=D6EIKDGF6ZWM76AACFCVQTSPKOKHBRKUIAMFTHQI6ZMGK4FXKOUA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:18.399477604 (566h5m38.891910047s) [noise=true , address=145.79.227.19:4079 , piece_id=7ZQS2H3PLGW225NRTL2HKWEOG5KJQ6YE3FKKROBZSTCZT53VCXRA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: rpc: dial tcp 145.79.227.19:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:18.399545556 (566h5m38.891977985s) [noise=true , address=145.79.227.7:4295 , piece_id=4HX2RRSRWSFJABQNL4EPSWLSYT24YVSAROWLJ4AKJEUM76VXLJZA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: rpc: dial tcp 145.79.227.7:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:18.399589040 (566h5m38.892021482s) [noise=true , address=145.79.227.13:4804 , piece_id=FCXCLNRFJN7WXVGMVR5KIXCV5CBYYJK42BZM466QEJ4KRGEIBE5A, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op Sep 22 04:13:18.399613607 (566h5m38.892046035s) [noise=true , address=145.79.227.3:4151 , piece_id=MRIW4AC56ZE4HZU3C7UPNZKLOFAQVK4W42K6BSEAVW3TBWP3M5BA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:18.399645674 (566h5m38.892078118s) [seg_pos={0 0}, upload=498007]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 22 04:13:18.514434154 (566h5m39.006866604s) [upload=498007]: Done uploading segments... Sep 22 04:13:18.514474162 (566h5m39.006906591s) [upload=498007]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 2.461192159s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....X.X.X..................X..X..XX..X......XXX.....XX.X..X] duration: 30.117357796s FAIL (2025-09-22 04:13:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== starting upload Sep 22 04:12:48.706801370 (566h5m9.199233813s) [upload=498007]: Uploading segments... Sep 22 04:12:48.707229293 (566h5m9.199661737s) [upload=498007]: Got next segment. Inline: false Sep 22 04:12:48.707297923 (566h5m9.199730364s) [upload=498007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:12:48.707306317 (566h5m9.199738764s) [upload=498007]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:12:48.892901897 (566h5m9.38533434s) [seg_pos={0 0}, upload=498007]: Begin upload segment... Sep 22 04:12:48.892949939 (566h5m9.385382371s) [seg_pos={0 0}, upload=498007]: Done begin upload segment. Sep 22 04:12:48.892975277 (566h5m9.38540772s) [noise=true , address=145.79.227.16:4909 , piece_id=U7LWV57HSE5QKQ7UU4I2AD7NX2B2ISJZUM5AUSK3OGBPA4ICGQ7A, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.892996534 (566h5m9.385428985s) [noise=true , address=145.79.227.3:4151 , piece_id=MRIW4AC56ZE4HZU3C7UPNZKLOFAQVK4W42K6BSEAVW3TBWP3M5BA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.893370435 (566h5m9.385802868s) [noise=true , address=145.79.227.5:4223 , piece_id=ABJ62GABVLVJJROKLPAK52PUFQJL5ECLOEEFNKGBFHLV4WFESP5A, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.894484015 (566h5m9.386916463s) [noise=true , address=145.79.227.5:4221 , piece_id=E4BKEVYABCE7XNYHGVL327IMJ4NVVSDVJKMO2KZSC5I2SSDNAWQQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.895012848 (566h5m9.387445305s) [noise=true , address=45.91.116.11:4117 , piece_id=J3EJ4A564X6SKQY74VLGP7LOGUJQANLHDU2PZR5KZ4N5CDYIGTVQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.896223753 (566h5m9.388656205s) [noise=true , address=145.79.227.4:4187 , piece_id=D6EIKDGF6ZWM76AACFCVQTSPKOKHBRKUIAMFTHQI6ZMGK4FXKOUA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.897278701 (566h5m9.389711153s) [noise=true , address=45.91.116.7:4010 , piece_id=VLKLHFG5UFNGAQCF7T6XS6AQ6EH7FR32BCUVJG4IWZBLGZVTWGYQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.898535819 (566h5m9.390968272s) [noise=true , address=145.79.227.13:4804 , piece_id=FCXCLNRFJN7WXVGMVR5KIXCV5CBYYJK42BZM466QEJ4KRGEIBE5A, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.899251154 (566h5m9.391683606s) [noise=true , address=145.79.227.11:4731 , piece_id=UINF5M2UA5JF5L25X3KGFDLBHQAHTDYAWWVSS67ZPZU4YPDUPJAQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.900606817 (566h5m9.393039269s) [noise=true , address=45.91.116.4:4939 , piece_id=SOAXVODPU4IAOCAUAFRERMBDOV6KX2QWR6VHBIN45HZF522X36NA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.901103197 (566h5m9.393535649s) [noise=true , address=45.91.116.9:4263 , piece_id=6NMFEKPW3VBWILRUE2TSSQIKKRZMK6QE2UB3IVG6B6ZZTLJBOIIQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.901505931 (566h5m9.393938382s) [noise=true , address=45.91.116.5:4297 , piece_id=Q3I55TEQKGYKFOZXTFAHFOCUD6UID7CTFQPBL444XKQKSNFFKTIQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.901953049 (566h5m9.394385501s) [noise=true , address=145.79.227.7:4295 , piece_id=4HX2RRSRWSFJABQNL4EPSWLSYT24YVSAROWLJ4AKJEUM76VXLJZA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.902444972 (566h5m9.394877424s) [noise=true , address=45.91.116.10:4081 , piece_id=IRDNUJENCOCKRS2TYGOO5ZTLMBQACK2ACMEZPZCHK24PTDLPYLVQ, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.903683159 (566h5m9.39611561s) [noise=true , address=77.90.31.15:4697 , piece_id=K7Q5ZEIO4CHNY447GTNJTWZA67WFQEEHYUHHY6N43ATQZAOPJQ7A, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:48.904162769 (566h5m9.396595222s) [noise=true , address=145.79.227.19:4079 , piece_id=7ZQS2H3PLGW225NRTL2HKWEOG5KJQ6YE3FKKROBZSTCZT53VCXRA, seg_pos={0 0}, upload=498007]: Uploading piece... Sep 22 04:12:49.516244091 (566h5m10.008676525s) [upload=498007]: Next returned nil segment Sep 22 04:12:49.516265465 (566h5m10.008697907s) [upload=498007]: Waiting for error group managing segments... Sep 22 04:12:49.654197031 (566h5m10.14662948s) [noise=true , address=45.91.116.4:4939 , piece_id=SOAXVODPU4IAOCAUAFRERMBDOV6KX2QWR6VHBIN45HZF522X36NA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.658411093 (566h5m10.15084353s) [noise=true , address=45.91.116.10:4081 , piece_id=IRDNUJENCOCKRS2TYGOO5ZTLMBQACK2ACMEZPZCHK24PTDLPYLVQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.658650347 (566h5m10.15108279s) [noise=true , address=77.90.31.15:4697 , piece_id=K7Q5ZEIO4CHNY447GTNJTWZA67WFQEEHYUHHY6N43ATQZAOPJQ7A, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.659224641 (566h5m10.151657075s) [noise=true , address=45.91.116.7:4010 , piece_id=VLKLHFG5UFNGAQCF7T6XS6AQ6EH7FR32BCUVJG4IWZBLGZVTWGYQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.660546209 (566h5m10.152978647s) [noise=true , address=45.91.116.5:4297 , piece_id=Q3I55TEQKGYKFOZXTFAHFOCUD6UID7CTFQPBL444XKQKSNFFKTIQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.667064175 (566h5m10.159496609s) [noise=true , address=45.91.116.9:4263 , piece_id=6NMFEKPW3VBWILRUE2TSSQIKKRZMK6QE2UB3IVG6B6ZZTLJBOIIQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:49.668979943 (566h5m10.161412378s) [noise=true , address=45.91.116.11:4117 , piece_id=J3EJ4A564X6SKQY74VLGP7LOGUJQANLHDU2PZR5KZ4N5CDYIGTVQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: <nil> Sep 22 04:12:50.852999799 (566h5m11.345432232s) [noise=true , address=145.79.227.11:4731 , piece_id=UINF5M2UA5JF5L25X3KGFDLBHQAHTDYAWWVSS67ZPZU4YPDUPJAQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:04.965629969 (566h5m25.458062418s) [noise=true , address=145.79.227.5:4223 , piece_id=ABJ62GABVLVJJROKLPAK52PUFQJL5ECLOEEFNKGBFHLV4WFESP5A, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:04.965895803 (566h5m25.458328236s) [noise=true , address=145.79.227.5:4221 , piece_id=E4BKEVYABCE7XNYHGVL327IMJ4NVVSDVJKMO2KZSC5I2SSDNAWQQ, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: rpc: dial tcp 145.79.227.5:4221: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:05.000653869 (566h5m25.49308636s) [noise=true , address=145.79.227.16:4909 , piece_id=U7LWV57HSE5QKQ7UU4I2AD7NX2B2ISJZUM5AUSK3OGBPA4ICGQ7A, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 04:13:18.399424194 (566h5m38.891856637s) [noise=true , address=145.79.227.4:4187 , piece_id=D6EIKDGF6ZWM76AACFCVQTSPKOKHBRKUIAMFTHQI6ZMGK4FXKOUA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:18.399477604 (566h5m38.891910047s) [noise=true , address=145.79.227.19:4079 , piece_id=7ZQS2H3PLGW225NRTL2HKWEOG5KJQ6YE3FKKROBZSTCZT53VCXRA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: rpc: dial tcp 145.79.227.19:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:18.399545556 (566h5m38.891977985s) [noise=true , address=145.79.227.7:4295 , piece_id=4HX2RRSRWSFJABQNL4EPSWLSYT24YVSAROWLJ4AKJEUM76VXLJZA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: rpc: dial tcp 145.79.227.7:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:18.399589040 (566h5m38.892021482s) [noise=true , address=145.79.227.13:4804 , piece_id=FCXCLNRFJN7WXVGMVR5KIXCV5CBYYJK42BZM466QEJ4KRGEIBE5A, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op Sep 22 04:13:18.399613607 (566h5m38.892046035s) [noise=true , address=145.79.227.3:4151 , piece_id=MRIW4AC56ZE4HZU3C7UPNZKLOFAQVK4W42K6BSEAVW3TBWP3M5BA, seg_pos={0 0}, upload=498007]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:13:18.399645674 (566h5m38.892078118s) [seg_pos={0 0}, upload=498007]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 22 04:13:18.514434154 (566h5m39.006866604s) [upload=498007]: Done uploading segments... Sep 22 04:13:18.514474162 (566h5m39.006906591s) [upload=498007]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.1173942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......X.X.X..................X..X..XX..X......XXX.....XX.X..] duration: 2.690729491s FAIL (2025-09-22 04:12:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:35.783885701 (566h2m56.276318155s) [upload=497973]: Starting upload Sep 22 04:10:35.783989936 (566h2m56.276422385s) [upload=497973]: Done starting upload Sep 22 04:10:35.784395864 (566h2m56.276828307s) [upload=497973]: Uploading segments... Sep 22 04:10:35.784454470 (566h2m56.276886928s) [upload=497973]: Got next segment. Inline: false Sep 22 04:10:35.784502028 (566h2m56.276934517s) [upload=497973]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:35.784512121 (566h2m56.27694457s) [upload=497973]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:35.902351134 (566h2m56.394783563s) [seg_pos={0 0}, upload=497973]: Begin upload segment... Sep 22 04:10:35.902413754 (566h2m56.394846197s) [seg_pos={0 0}, upload=497973]: Done begin upload segment. Sep 22 04:10:35.902462333 (566h2m56.394894781s) [noise=true , address=145.79.227.4:4185 , piece_id=XJ574EAXK7VTBPJNXKA3UWMOLHECWR4DDQLFADPGXC3VD2Z74MUQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.902485943 (566h2m56.394918408s) [noise=true , address=145.79.227.13:4802 , piece_id=VJ74XS26T5A52NYXX3AWR6YM37LKBX3B3663TEQHIJS65K5SHRKA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.902556520 (566h2m56.39498897s) [noise=true , address=145.79.227.5:4222 , piece_id=2XIQYY4EQPJOFWPEF2R43SSKKY3SVGEWXHMVQUTAXUM6P2QLDKEA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903012567 (566h2m56.395445017s) [noise=true , address=77.90.31.12:4592 , piece_id=DCHVPUITHP3FTL6TV4U7W7TIIXGLDFMRXBG56EFKPLM5MZU7WYQA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903084250 (566h2m56.395516702s) [noise=true , address=145.79.227.7:4293 , piece_id=SGJSO7N5C7PEVHXR7EK55KTEHT7ECQAOTPLX6SXIHE2WILEBYLFA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903487679 (566h2m56.39592013s) [noise=true , address=77.90.31.15:4699 , piece_id=YEO2ICKHQZGS6NKVVZ737KWHTDJSS4PMKSSHHO6HZWW6JAMXGQ3A, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903630419 (566h2m56.396062871s) [noise=true , address=145.79.227.12:4767 , piece_id=UV3S6CDOIFJ4YM7ZVEWEVG5GLUDWWGSH3KREWYJGJVCMHYZ3Z23A, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903937472 (566h2m56.396369921s) [noise=true , address=45.91.116.13:4155 , piece_id=5JRPBMBPPYN6UPXXXS35KMTFPC47XLIKIWVBAKRCOHDT326U7QRQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904101251 (566h2m56.396533704s) [noise=true , address=45.91.116.9:4261 , piece_id=JXSYQ3QJO4ADSLTPWNSYIBBCJWLQLD2NU2JQDYAB3HLVMDPFATPQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904311039 (566h2m56.396743484s) [noise=true , address=45.91.116.13:4153 , piece_id=5PZRX6KGP6QGVPP43EENBNNOGGZE5S4BVXN5OEJDJSCIOZEJNAOQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904650890 (566h2m56.397083343s) [noise=true , address=77.90.31.17:4770 , piece_id=4N2NM7LQAKFMR6LD4VFRR6TZXHROOVMPFTXG5DQDZYFT2X3AFJBQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.905108970 (566h2m56.397541423s) [noise=true , address=77.90.31.14:4661 , piece_id=PUOWNUN5PDYW5R2KYETNAAAQ7UPKFFYR5XDNKU4EMF5DXPPJSJ4Q, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.905531010 (566h2m56.397963461s) [noise=true , address=77.90.31.13:4628 , piece_id=HJWKM6NRIRHDZV5TEPTVWKDV5EILERRC7XGFBVMM2NQ6673T35EQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.906492009 (566h2m56.398924475s) [noise=true , address=77.90.31.21:4878 , piece_id=CBQU5ICANCVG2AZQSVFZ6ARMNLK6X57HKGQ4XZJRW2HVHZ6OKL7Q, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.907248421 (566h2m56.399680874s) [noise=true , address=145.79.227.18:4005 , piece_id=CRLKNRJJI2TNJPXXPNIFX77GKNFCMOP5HXMDF5R6IIKOLXKUTMYQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.908334248 (566h2m56.400766715s) [noise=true , address=77.90.31.19:4842 , piece_id=ZP2GI3QIPDCXRWAOI2C46Y5VKKOBWRRYCI4L2LSRCFBKJUISM4MQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:36.525004616 (566h2m57.017437051s) [upload=497973]: Next returned nil segment Sep 22 04:10:36.525024557 (566h2m57.017457s) [upload=497973]: Waiting for error group managing segments... Sep 22 04:10:36.653776442 (566h2m57.146208878s) [noise=true , address=77.90.31.12:4592 , piece_id=DCHVPUITHP3FTL6TV4U7W7TIIXGLDFMRXBG56EFKPLM5MZU7WYQA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.654570708 (566h2m57.147003141s) [noise=true , address=77.90.31.14:4661 , piece_id=PUOWNUN5PDYW5R2KYETNAAAQ7UPKFFYR5XDNKU4EMF5DXPPJSJ4Q, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.656213023 (566h2m57.148645463s) [noise=true , address=77.90.31.13:4628 , piece_id=HJWKM6NRIRHDZV5TEPTVWKDV5EILERRC7XGFBVMM2NQ6673T35EQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.656702138 (566h2m57.149134567s) [noise=true , address=77.90.31.17:4770 , piece_id=4N2NM7LQAKFMR6LD4VFRR6TZXHROOVMPFTXG5DQDZYFT2X3AFJBQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.657184255 (566h2m57.149616692s) [noise=true , address=77.90.31.21:4878 , piece_id=CBQU5ICANCVG2AZQSVFZ6ARMNLK6X57HKGQ4XZJRW2HVHZ6OKL7Q, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.657687378 (566h2m57.150119813s) [noise=true , address=45.91.116.9:4261 , piece_id=JXSYQ3QJO4ADSLTPWNSYIBBCJWLQLD2NU2JQDYAB3HLVMDPFATPQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.670564000 (566h2m57.162996435s) [noise=true , address=45.91.116.13:4153 , piece_id=5PZRX6KGP6QGVPP43EENBNNOGGZE5S4BVXN5OEJDJSCIOZEJNAOQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.672899453 (566h2m57.165331888s) [noise=true , address=77.90.31.19:4842 , piece_id=ZP2GI3QIPDCXRWAOI2C46Y5VKKOBWRRYCI4L2LSRCFBKJUISM4MQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.678768088 (566h2m57.171200521s) [noise=true , address=77.90.31.15:4699 , piece_id=YEO2ICKHQZGS6NKVVZ737KWHTDJSS4PMKSSHHO6HZWW6JAMXGQ3A, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.678827604 (566h2m57.171260039s) [noise=true , address=45.91.116.13:4155 , piece_id=5JRPBMBPPYN6UPXXXS35KMTFPC47XLIKIWVBAKRCOHDT326U7QRQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:11:05.478836640 (566h3m25.971269082s) [noise=true , address=145.79.227.13:4802 , piece_id=VJ74XS26T5A52NYXX3AWR6YM37LKBX3B3663TEQHIJS65K5SHRKA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.478923534 (566h3m25.971355977s) [noise=true , address=145.79.227.5:4222 , piece_id=2XIQYY4EQPJOFWPEF2R43SSKKY3SVGEWXHMVQUTAXUM6P2QLDKEA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479010878 (566h3m25.971443314s) [noise=true , address=145.79.227.12:4767 , piece_id=UV3S6CDOIFJ4YM7ZVEWEVG5GLUDWWGSH3KREWYJGJVCMHYZ3Z23A, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:11:05.479064451 (566h3m25.971496894s) [noise=true , address=145.79.227.18:4005 , piece_id=CRLKNRJJI2TNJPXXPNIFX77GKNFCMOP5HXMDF5R6IIKOLXKUTMYQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479143539 (566h3m25.971576012s) [noise=true , address=145.79.227.4:4185 , piece_id=XJ574EAXK7VTBPJNXKA3UWMOLHECWR4DDQLFADPGXC3VD2Z74MUQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479170355 (566h3m25.971602784s) [noise=true , address=145.79.227.7:4293 , piece_id=SGJSO7N5C7PEVHXR7EK55KTEHT7ECQAOTPLX6SXIHE2WILEBYLFA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479196424 (566h3m25.971628877s) [seg_pos={0 0}, upload=497973]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:11:05.589749082 (566h3m26.082181518s) [upload=497973]: Done uploading segments... Sep 22 04:11:05.589782234 (566h3m26.082214681s) [upload=497973]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.690761884s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.XX..........X.......X......X..XXXX.........................] duration: 2.775653424s FAIL (2025-09-22 04:12:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.775688292s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XX..........X.......X......X..XXXX........................] duration: 2.799210116s FAIL (2025-09-22 04:12:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.799239442s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......X.X.X..................X..X..XX..X......XXX.....XX.X.] duration: 2.601910978s FAIL (2025-09-22 04:11:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:35.783885701 (566h2m56.276318155s) [upload=497973]: Starting upload Sep 22 04:10:35.783989936 (566h2m56.276422385s) [upload=497973]: Done starting upload Sep 22 04:10:35.784395864 (566h2m56.276828307s) [upload=497973]: Uploading segments... Sep 22 04:10:35.784454470 (566h2m56.276886928s) [upload=497973]: Got next segment. Inline: false Sep 22 04:10:35.784502028 (566h2m56.276934517s) [upload=497973]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:35.784512121 (566h2m56.27694457s) [upload=497973]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:35.902351134 (566h2m56.394783563s) [seg_pos={0 0}, upload=497973]: Begin upload segment... Sep 22 04:10:35.902413754 (566h2m56.394846197s) [seg_pos={0 0}, upload=497973]: Done begin upload segment. Sep 22 04:10:35.902462333 (566h2m56.394894781s) [noise=true , address=145.79.227.4:4185 , piece_id=XJ574EAXK7VTBPJNXKA3UWMOLHECWR4DDQLFADPGXC3VD2Z74MUQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.902485943 (566h2m56.394918408s) [noise=true , address=145.79.227.13:4802 , piece_id=VJ74XS26T5A52NYXX3AWR6YM37LKBX3B3663TEQHIJS65K5SHRKA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.902556520 (566h2m56.39498897s) [noise=true , address=145.79.227.5:4222 , piece_id=2XIQYY4EQPJOFWPEF2R43SSKKY3SVGEWXHMVQUTAXUM6P2QLDKEA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903012567 (566h2m56.395445017s) [noise=true , address=77.90.31.12:4592 , piece_id=DCHVPUITHP3FTL6TV4U7W7TIIXGLDFMRXBG56EFKPLM5MZU7WYQA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903084250 (566h2m56.395516702s) [noise=true , address=145.79.227.7:4293 , piece_id=SGJSO7N5C7PEVHXR7EK55KTEHT7ECQAOTPLX6SXIHE2WILEBYLFA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903487679 (566h2m56.39592013s) [noise=true , address=77.90.31.15:4699 , piece_id=YEO2ICKHQZGS6NKVVZ737KWHTDJSS4PMKSSHHO6HZWW6JAMXGQ3A, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903630419 (566h2m56.396062871s) [noise=true , address=145.79.227.12:4767 , piece_id=UV3S6CDOIFJ4YM7ZVEWEVG5GLUDWWGSH3KREWYJGJVCMHYZ3Z23A, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903937472 (566h2m56.396369921s) [noise=true , address=45.91.116.13:4155 , piece_id=5JRPBMBPPYN6UPXXXS35KMTFPC47XLIKIWVBAKRCOHDT326U7QRQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904101251 (566h2m56.396533704s) [noise=true , address=45.91.116.9:4261 , piece_id=JXSYQ3QJO4ADSLTPWNSYIBBCJWLQLD2NU2JQDYAB3HLVMDPFATPQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904311039 (566h2m56.396743484s) [noise=true , address=45.91.116.13:4153 , piece_id=5PZRX6KGP6QGVPP43EENBNNOGGZE5S4BVXN5OEJDJSCIOZEJNAOQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904650890 (566h2m56.397083343s) [noise=true , address=77.90.31.17:4770 , piece_id=4N2NM7LQAKFMR6LD4VFRR6TZXHROOVMPFTXG5DQDZYFT2X3AFJBQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.905108970 (566h2m56.397541423s) [noise=true , address=77.90.31.14:4661 , piece_id=PUOWNUN5PDYW5R2KYETNAAAQ7UPKFFYR5XDNKU4EMF5DXPPJSJ4Q, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.905531010 (566h2m56.397963461s) [noise=true , address=77.90.31.13:4628 , piece_id=HJWKM6NRIRHDZV5TEPTVWKDV5EILERRC7XGFBVMM2NQ6673T35EQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.906492009 (566h2m56.398924475s) [noise=true , address=77.90.31.21:4878 , piece_id=CBQU5ICANCVG2AZQSVFZ6ARMNLK6X57HKGQ4XZJRW2HVHZ6OKL7Q, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.907248421 (566h2m56.399680874s) [noise=true , address=145.79.227.18:4005 , piece_id=CRLKNRJJI2TNJPXXPNIFX77GKNFCMOP5HXMDF5R6IIKOLXKUTMYQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.908334248 (566h2m56.400766715s) [noise=true , address=77.90.31.19:4842 , piece_id=ZP2GI3QIPDCXRWAOI2C46Y5VKKOBWRRYCI4L2LSRCFBKJUISM4MQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:36.525004616 (566h2m57.017437051s) [upload=497973]: Next returned nil segment Sep 22 04:10:36.525024557 (566h2m57.017457s) [upload=497973]: Waiting for error group managing segments... Sep 22 04:10:36.653776442 (566h2m57.146208878s) [noise=true , address=77.90.31.12:4592 , piece_id=DCHVPUITHP3FTL6TV4U7W7TIIXGLDFMRXBG56EFKPLM5MZU7WYQA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.654570708 (566h2m57.147003141s) [noise=true , address=77.90.31.14:4661 , piece_id=PUOWNUN5PDYW5R2KYETNAAAQ7UPKFFYR5XDNKU4EMF5DXPPJSJ4Q, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.656213023 (566h2m57.148645463s) [noise=true , address=77.90.31.13:4628 , piece_id=HJWKM6NRIRHDZV5TEPTVWKDV5EILERRC7XGFBVMM2NQ6673T35EQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.656702138 (566h2m57.149134567s) [noise=true , address=77.90.31.17:4770 , piece_id=4N2NM7LQAKFMR6LD4VFRR6TZXHROOVMPFTXG5DQDZYFT2X3AFJBQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.657184255 (566h2m57.149616692s) [noise=true , address=77.90.31.21:4878 , piece_id=CBQU5ICANCVG2AZQSVFZ6ARMNLK6X57HKGQ4XZJRW2HVHZ6OKL7Q, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.657687378 (566h2m57.150119813s) [noise=true , address=45.91.116.9:4261 , piece_id=JXSYQ3QJO4ADSLTPWNSYIBBCJWLQLD2NU2JQDYAB3HLVMDPFATPQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.670564000 (566h2m57.162996435s) [noise=true , address=45.91.116.13:4153 , piece_id=5PZRX6KGP6QGVPP43EENBNNOGGZE5S4BVXN5OEJDJSCIOZEJNAOQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.672899453 (566h2m57.165331888s) [noise=true , address=77.90.31.19:4842 , piece_id=ZP2GI3QIPDCXRWAOI2C46Y5VKKOBWRRYCI4L2LSRCFBKJUISM4MQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.678768088 (566h2m57.171200521s) [noise=true , address=77.90.31.15:4699 , piece_id=YEO2ICKHQZGS6NKVVZ737KWHTDJSS4PMKSSHHO6HZWW6JAMXGQ3A, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.678827604 (566h2m57.171260039s) [noise=true , address=45.91.116.13:4155 , piece_id=5JRPBMBPPYN6UPXXXS35KMTFPC47XLIKIWVBAKRCOHDT326U7QRQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:11:05.478836640 (566h3m25.971269082s) [noise=true , address=145.79.227.13:4802 , piece_id=VJ74XS26T5A52NYXX3AWR6YM37LKBX3B3663TEQHIJS65K5SHRKA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.478923534 (566h3m25.971355977s) [noise=true , address=145.79.227.5:4222 , piece_id=2XIQYY4EQPJOFWPEF2R43SSKKY3SVGEWXHMVQUTAXUM6P2QLDKEA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479010878 (566h3m25.971443314s) [noise=true , address=145.79.227.12:4767 , piece_id=UV3S6CDOIFJ4YM7ZVEWEVG5GLUDWWGSH3KREWYJGJVCMHYZ3Z23A, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:11:05.479064451 (566h3m25.971496894s) [noise=true , address=145.79.227.18:4005 , piece_id=CRLKNRJJI2TNJPXXPNIFX77GKNFCMOP5HXMDF5R6IIKOLXKUTMYQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479143539 (566h3m25.971576012s) [noise=true , address=145.79.227.4:4185 , piece_id=XJ574EAXK7VTBPJNXKA3UWMOLHECWR4DDQLFADPGXC3VD2Z74MUQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479170355 (566h3m25.971602784s) [noise=true , address=145.79.227.7:4293 , piece_id=SGJSO7N5C7PEVHXR7EK55KTEHT7ECQAOTPLX6SXIHE2WILEBYLFA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479196424 (566h3m25.971628877s) [seg_pos={0 0}, upload=497973]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:11:05.589749082 (566h3m26.082181518s) [upload=497973]: Done uploading segments... Sep 22 04:11:05.589782234 (566h3m26.082214681s) [upload=497973]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.601941862s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [........X.X.X..................X..X..XX..X......XXX.....XX.X] duration: 30.112678036s FAIL (2025-09-22 04:11:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:35.783885701 (566h2m56.276318155s) [upload=497973]: Starting upload Sep 22 04:10:35.783989936 (566h2m56.276422385s) [upload=497973]: Done starting upload Sep 22 04:10:35.784395864 (566h2m56.276828307s) [upload=497973]: Uploading segments... Sep 22 04:10:35.784454470 (566h2m56.276886928s) [upload=497973]: Got next segment. Inline: false Sep 22 04:10:35.784502028 (566h2m56.276934517s) [upload=497973]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:35.784512121 (566h2m56.27694457s) [upload=497973]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:35.902351134 (566h2m56.394783563s) [seg_pos={0 0}, upload=497973]: Begin upload segment... Sep 22 04:10:35.902413754 (566h2m56.394846197s) [seg_pos={0 0}, upload=497973]: Done begin upload segment. Sep 22 04:10:35.902462333 (566h2m56.394894781s) [noise=true , address=145.79.227.4:4185 , piece_id=XJ574EAXK7VTBPJNXKA3UWMOLHECWR4DDQLFADPGXC3VD2Z74MUQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.902485943 (566h2m56.394918408s) [noise=true , address=145.79.227.13:4802 , piece_id=VJ74XS26T5A52NYXX3AWR6YM37LKBX3B3663TEQHIJS65K5SHRKA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.902556520 (566h2m56.39498897s) [noise=true , address=145.79.227.5:4222 , piece_id=2XIQYY4EQPJOFWPEF2R43SSKKY3SVGEWXHMVQUTAXUM6P2QLDKEA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903012567 (566h2m56.395445017s) [noise=true , address=77.90.31.12:4592 , piece_id=DCHVPUITHP3FTL6TV4U7W7TIIXGLDFMRXBG56EFKPLM5MZU7WYQA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903084250 (566h2m56.395516702s) [noise=true , address=145.79.227.7:4293 , piece_id=SGJSO7N5C7PEVHXR7EK55KTEHT7ECQAOTPLX6SXIHE2WILEBYLFA, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903487679 (566h2m56.39592013s) [noise=true , address=77.90.31.15:4699 , piece_id=YEO2ICKHQZGS6NKVVZ737KWHTDJSS4PMKSSHHO6HZWW6JAMXGQ3A, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903630419 (566h2m56.396062871s) [noise=true , address=145.79.227.12:4767 , piece_id=UV3S6CDOIFJ4YM7ZVEWEVG5GLUDWWGSH3KREWYJGJVCMHYZ3Z23A, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.903937472 (566h2m56.396369921s) [noise=true , address=45.91.116.13:4155 , piece_id=5JRPBMBPPYN6UPXXXS35KMTFPC47XLIKIWVBAKRCOHDT326U7QRQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904101251 (566h2m56.396533704s) [noise=true , address=45.91.116.9:4261 , piece_id=JXSYQ3QJO4ADSLTPWNSYIBBCJWLQLD2NU2JQDYAB3HLVMDPFATPQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904311039 (566h2m56.396743484s) [noise=true , address=45.91.116.13:4153 , piece_id=5PZRX6KGP6QGVPP43EENBNNOGGZE5S4BVXN5OEJDJSCIOZEJNAOQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.904650890 (566h2m56.397083343s) [noise=true , address=77.90.31.17:4770 , piece_id=4N2NM7LQAKFMR6LD4VFRR6TZXHROOVMPFTXG5DQDZYFT2X3AFJBQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.905108970 (566h2m56.397541423s) [noise=true , address=77.90.31.14:4661 , piece_id=PUOWNUN5PDYW5R2KYETNAAAQ7UPKFFYR5XDNKU4EMF5DXPPJSJ4Q, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.905531010 (566h2m56.397963461s) [noise=true , address=77.90.31.13:4628 , piece_id=HJWKM6NRIRHDZV5TEPTVWKDV5EILERRC7XGFBVMM2NQ6673T35EQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.906492009 (566h2m56.398924475s) [noise=true , address=77.90.31.21:4878 , piece_id=CBQU5ICANCVG2AZQSVFZ6ARMNLK6X57HKGQ4XZJRW2HVHZ6OKL7Q, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.907248421 (566h2m56.399680874s) [noise=true , address=145.79.227.18:4005 , piece_id=CRLKNRJJI2TNJPXXPNIFX77GKNFCMOP5HXMDF5R6IIKOLXKUTMYQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:35.908334248 (566h2m56.400766715s) [noise=true , address=77.90.31.19:4842 , piece_id=ZP2GI3QIPDCXRWAOI2C46Y5VKKOBWRRYCI4L2LSRCFBKJUISM4MQ, seg_pos={0 0}, upload=497973]: Uploading piece... Sep 22 04:10:36.525004616 (566h2m57.017437051s) [upload=497973]: Next returned nil segment Sep 22 04:10:36.525024557 (566h2m57.017457s) [upload=497973]: Waiting for error group managing segments... Sep 22 04:10:36.653776442 (566h2m57.146208878s) [noise=true , address=77.90.31.12:4592 , piece_id=DCHVPUITHP3FTL6TV4U7W7TIIXGLDFMRXBG56EFKPLM5MZU7WYQA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.654570708 (566h2m57.147003141s) [noise=true , address=77.90.31.14:4661 , piece_id=PUOWNUN5PDYW5R2KYETNAAAQ7UPKFFYR5XDNKU4EMF5DXPPJSJ4Q, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.656213023 (566h2m57.148645463s) [noise=true , address=77.90.31.13:4628 , piece_id=HJWKM6NRIRHDZV5TEPTVWKDV5EILERRC7XGFBVMM2NQ6673T35EQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.656702138 (566h2m57.149134567s) [noise=true , address=77.90.31.17:4770 , piece_id=4N2NM7LQAKFMR6LD4VFRR6TZXHROOVMPFTXG5DQDZYFT2X3AFJBQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.657184255 (566h2m57.149616692s) [noise=true , address=77.90.31.21:4878 , piece_id=CBQU5ICANCVG2AZQSVFZ6ARMNLK6X57HKGQ4XZJRW2HVHZ6OKL7Q, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.657687378 (566h2m57.150119813s) [noise=true , address=45.91.116.9:4261 , piece_id=JXSYQ3QJO4ADSLTPWNSYIBBCJWLQLD2NU2JQDYAB3HLVMDPFATPQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.670564000 (566h2m57.162996435s) [noise=true , address=45.91.116.13:4153 , piece_id=5PZRX6KGP6QGVPP43EENBNNOGGZE5S4BVXN5OEJDJSCIOZEJNAOQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.672899453 (566h2m57.165331888s) [noise=true , address=77.90.31.19:4842 , piece_id=ZP2GI3QIPDCXRWAOI2C46Y5VKKOBWRRYCI4L2LSRCFBKJUISM4MQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.678768088 (566h2m57.171200521s) [noise=true , address=77.90.31.15:4699 , piece_id=YEO2ICKHQZGS6NKVVZ737KWHTDJSS4PMKSSHHO6HZWW6JAMXGQ3A, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:10:36.678827604 (566h2m57.171260039s) [noise=true , address=45.91.116.13:4155 , piece_id=5JRPBMBPPYN6UPXXXS35KMTFPC47XLIKIWVBAKRCOHDT326U7QRQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: <nil> Sep 22 04:11:05.478836640 (566h3m25.971269082s) [noise=true , address=145.79.227.13:4802 , piece_id=VJ74XS26T5A52NYXX3AWR6YM37LKBX3B3663TEQHIJS65K5SHRKA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.478923534 (566h3m25.971355977s) [noise=true , address=145.79.227.5:4222 , piece_id=2XIQYY4EQPJOFWPEF2R43SSKKY3SVGEWXHMVQUTAXUM6P2QLDKEA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479010878 (566h3m25.971443314s) [noise=true , address=145.79.227.12:4767 , piece_id=UV3S6CDOIFJ4YM7ZVEWEVG5GLUDWWGSH3KREWYJGJVCMHYZ3Z23A, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:11:05.479064451 (566h3m25.971496894s) [noise=true , address=145.79.227.18:4005 , piece_id=CRLKNRJJI2TNJPXXPNIFX77GKNFCMOP5HXMDF5R6IIKOLXKUTMYQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479143539 (566h3m25.971576012s) [noise=true , address=145.79.227.4:4185 , piece_id=XJ574EAXK7VTBPJNXKA3UWMOLHECWR4DDQLFADPGXC3VD2Z74MUQ, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479170355 (566h3m25.971602784s) [noise=true , address=145.79.227.7:4293 , piece_id=SGJSO7N5C7PEVHXR7EK55KTEHT7ECQAOTPLX6SXIHE2WILEBYLFA, seg_pos={0 0}, upload=497973]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:11:05.479196424 (566h3m25.971628877s) [seg_pos={0 0}, upload=497973]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:11:05.589749082 (566h3m26.082181518s) [upload=497973]: Done uploading segments... Sep 22 04:11:05.589782234 (566h3m26.082214681s) [upload=497973]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.112707178s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...XX..........X.......X......X..XXXX.......................] duration: 2.746319739s FAIL (2025-09-22 04:10:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.746351601s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X...XX..........X.......X......X..XXXX......................] duration: 2.676892221s FAIL (2025-09-22 04:10:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.677182015s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X........X.X.X..................X..X..XX..X......XXX.....XX.] duration: 2.435082674s FAIL (2025-09-22 04:09:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 04:08:20.092873855 (566h0m40.585306309s) [upload=497944]: Starting upload Sep 22 04:08:20.092980012 (566h0m40.585412464s) [upload=497944]: Done starting upload Sep 22 04:08:20.093125272 (566h0m40.585557708s) [upload=497944]: Uploading segments... Sep 22 04:08:20.093406697 (566h0m40.585839128s) [upload=497944]: Got next segment. Inline: false Sep 22 04:08:20.093461574 (566h0m40.585894005s) [upload=497944]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:08:20.093467358 (566h0m40.5858998s) [upload=497944]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:08:20.214649782 (566h0m40.707082226s) [seg_pos={0 0}, upload=497944]: Begin upload segment... Sep 22 04:08:20.214696571 (566h0m40.707128999s) [seg_pos={0 0}, upload=497944]: Done begin upload segment. Sep 22 04:08:20.214738544 (566h0m40.707170992s) [noise=true , address=145.79.227.13:4801 , piece_id=EXUBHS3HT4ENMTYZTEE3UT7CHPZ4BB5FA3LU6U7FTCMQZTAQULUA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.215286567 (566h0m40.707719015s) [noise=true , address=77.90.31.21:4878 , piece_id=7CUHFWHWCSFLGPXWBGMQL5IESETHDUBZ7RH35V6RUSL3O66RUSOQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.215729597 (566h0m40.708162042s) [noise=true , address=145.79.227.6:4259 , piece_id=SU6SZNPMK5X2MY7X3J4W4QUX32VVHSBLG52KTQMQWJVKGMNXKBYA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.216046882 (566h0m40.708479327s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZMW5KITYPRPR5GGQOY6HQOBDBDO2PUWETTMU7ZPYHVSRYMYPFWA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.216133922 (566h0m40.708566368s) [noise=true , address=45.91.116.4:4939 , piece_id=WL2H25EZ5M6LF4VIGCNOL7OTCG27CCY5VPER7IXPTQ4GVW5SSFIQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.216682051 (566h0m40.709114496s) [noise=true , address=145.79.227.19:4078 , piece_id=UDKV5GIBVIZGRFCC3N3BYVWYYVU6QX7N2VXSZINHI5RB6SA2ZG4A, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.218314572 (566h0m40.710747024s) [noise=true , address=45.91.116.7:4009 , piece_id=V35EMPA6QQ32EA55FNTLVBTTA6DH3ALHKJXS562TPFEVUVMQJTPQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.218755483 (566h0m40.711187936s) [noise=true , address=45.91.116.12:4227 , piece_id=OI2H2C33CY7YZXT3GB2IM3UOZEATOPUUAVK5AHFO2MVWNGK6OYTA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.219126261 (566h0m40.711558713s) [noise=true , address=45.91.116.11:4117 , piece_id=GUPR25YGE6U5SK34JIUAA5UPDLK3CUJPNLL64T63MM4W6CJ2HVJA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.219492564 (566h0m40.711925017s) [noise=true , address=77.90.31.14:4662 , piece_id=DOETS5GWB5TETOY2GU5BURWDBX64HWEEKDNFLQ2ENNPUSQXNTIOQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.219845801 (566h0m40.712278254s) [noise=true , address=145.79.227.11:4729 , piece_id=U7JH37SW7YHQP6LPYEVSS2BNPDCW3G4O4VWPL6FXDYBEMDKUTBHQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.220230214 (566h0m40.71266267s) [noise=true , address=45.91.116.9:4263 , piece_id=EXDHF74U2IBVZNY74S4M23CLFYNUUAKVDPWDUV6Z6Q3AEGUQJTMQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.221088298 (566h0m40.71352075s) [noise=true , address=145.79.227.2:4115 , piece_id=BL5OA2EFGTPOCOAR7RRHZR2SNYICU6N2XUH5GFVASJ3NJ7RIUX3A, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.221678245 (566h0m40.714110697s) [noise=true , address=77.90.31.19:4841 , piece_id=CT33BOQA5SAHHXD5H5VO4XXNAGJXOO3JI4SF6T3L5ZZ5DXN24NXA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.222178603 (566h0m40.714611054s) [noise=true , address=145.79.227.3:4151 , piece_id=XYBCZQEYZ77IG62ZQVVAUOPLGRCUKJQWBOCNQFHUE5MJ4MM33EBA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.222503950 (566h0m40.714936392s) [noise=true , address=145.79.227.5:4221 , piece_id=D5YEIOQRAH6VQRNWO3AXOFEF6EEQYA2LLPNDZ6TYQFJ2WH57PDIA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.843905532 (566h0m41.336337975s) [upload=497944]: Next returned nil segment Sep 22 04:08:20.843924538 (566h0m41.336356983s) [upload=497944]: Waiting for error group managing segments... Sep 22 04:08:20.974955345 (566h0m41.467387782s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZMW5KITYPRPR5GGQOY6HQOBDBDO2PUWETTMU7ZPYHVSRYMYPFWA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.976288091 (566h0m41.46872052s) [noise=true , address=45.91.116.4:4939 , piece_id=WL2H25EZ5M6LF4VIGCNOL7OTCG27CCY5VPER7IXPTQ4GVW5SSFIQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.976377050 (566h0m41.468809486s) [noise=true , address=45.91.116.7:4009 , piece_id=V35EMPA6QQ32EA55FNTLVBTTA6DH3ALHKJXS562TPFEVUVMQJTPQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.976677762 (566h0m41.46911019s) [noise=true , address=77.90.31.19:4841 , piece_id=CT33BOQA5SAHHXD5H5VO4XXNAGJXOO3JI4SF6T3L5ZZ5DXN24NXA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.979438006 (566h0m41.471870442s) [noise=true , address=45.91.116.9:4263 , piece_id=EXDHF74U2IBVZNY74S4M23CLFYNUUAKVDPWDUV6Z6Q3AEGUQJTMQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.980005232 (566h0m41.472437661s) [noise=true , address=45.91.116.12:4227 , piece_id=OI2H2C33CY7YZXT3GB2IM3UOZEATOPUUAVK5AHFO2MVWNGK6OYTA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.982100099 (566h0m41.474532534s) [noise=true , address=77.90.31.14:4662 , piece_id=DOETS5GWB5TETOY2GU5BURWDBX64HWEEKDNFLQ2ENNPUSQXNTIOQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.982843008 (566h0m41.475275436s) [noise=true , address=45.91.116.11:4117 , piece_id=GUPR25YGE6U5SK34JIUAA5UPDLK3CUJPNLL64T63MM4W6CJ2HVJA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.986232374 (566h0m41.478664805s) [noise=true , address=77.90.31.21:4878 , piece_id=7CUHFWHWCSFLGPXWBGMQL5IESETHDUBZ7RH35V6RUSL3O66RUSOQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:49.779601525 (566h1m10.272033958s) [noise=true , address=145.79.227.5:4221 , piece_id=D5YEIOQRAH6VQRNWO3AXOFEF6EEQYA2LLPNDZ6TYQFJ2WH57PDIA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: rpc: dial tcp 145.79.227.5:4221: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779670667 (566h1m10.272103099s) [noise=true , address=145.79.227.3:4151 , piece_id=XYBCZQEYZ77IG62ZQVVAUOPLGRCUKJQWBOCNQFHUE5MJ4MM33EBA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779695716 (566h1m10.272128149s) [noise=true , address=145.79.227.6:4259 , piece_id=SU6SZNPMK5X2MY7X3J4W4QUX32VVHSBLG52KTQMQWJVKGMNXKBYA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779851127 (566h1m10.272283563s) [noise=true , address=145.79.227.11:4729 , piece_id=U7JH37SW7YHQP6LPYEVSS2BNPDCW3G4O4VWPL6FXDYBEMDKUTBHQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779899437 (566h1m10.272331871s) [noise=true , address=145.79.227.19:4078 , piece_id=UDKV5GIBVIZGRFCC3N3BYVWYYVU6QX7N2VXSZINHI5RB6SA2ZG4A, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779910947 (566h1m10.272343382s) [noise=true , address=145.79.227.2:4115 , piece_id=BL5OA2EFGTPOCOAR7RRHZR2SNYICU6N2XUH5GFVASJ3NJ7RIUX3A, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779930053 (566h1m10.272362488s) [noise=true , address=145.79.227.13:4801 , piece_id=EXUBHS3HT4ENMTYZTEE3UT7CHPZ4BB5FA3LU6U7FTCMQZTAQULUA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779987109 (566h1m10.272419559s) [seg_pos={0 0}, upload=497944]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 04:08:49.900199100 (566h1m10.392631537s) [upload=497944]: Done uploading segments... Sep 22 04:08:49.900247349 (566h1m10.392679799s) [upload=497944]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.435134436s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X...XX..........X.......X......X..XXXX.....................] duration: 3.048455091s FAIL (2025-09-22 04:09:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.048495929s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [XX........X.X.X..................X..X..XX..X......XXX.....XX] duration: 30.122734109s FAIL (2025-09-22 04:08:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 04:08:20.092873855 (566h0m40.585306309s) [upload=497944]: Starting upload Sep 22 04:08:20.092980012 (566h0m40.585412464s) [upload=497944]: Done starting upload Sep 22 04:08:20.093125272 (566h0m40.585557708s) [upload=497944]: Uploading segments... Sep 22 04:08:20.093406697 (566h0m40.585839128s) [upload=497944]: Got next segment. Inline: false Sep 22 04:08:20.093461574 (566h0m40.585894005s) [upload=497944]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:08:20.093467358 (566h0m40.5858998s) [upload=497944]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:08:20.214649782 (566h0m40.707082226s) [seg_pos={0 0}, upload=497944]: Begin upload segment... Sep 22 04:08:20.214696571 (566h0m40.707128999s) [seg_pos={0 0}, upload=497944]: Done begin upload segment. Sep 22 04:08:20.214738544 (566h0m40.707170992s) [noise=true , address=145.79.227.13:4801 , piece_id=EXUBHS3HT4ENMTYZTEE3UT7CHPZ4BB5FA3LU6U7FTCMQZTAQULUA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.215286567 (566h0m40.707719015s) [noise=true , address=77.90.31.21:4878 , piece_id=7CUHFWHWCSFLGPXWBGMQL5IESETHDUBZ7RH35V6RUSL3O66RUSOQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.215729597 (566h0m40.708162042s) [noise=true , address=145.79.227.6:4259 , piece_id=SU6SZNPMK5X2MY7X3J4W4QUX32VVHSBLG52KTQMQWJVKGMNXKBYA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.216046882 (566h0m40.708479327s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZMW5KITYPRPR5GGQOY6HQOBDBDO2PUWETTMU7ZPYHVSRYMYPFWA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.216133922 (566h0m40.708566368s) [noise=true , address=45.91.116.4:4939 , piece_id=WL2H25EZ5M6LF4VIGCNOL7OTCG27CCY5VPER7IXPTQ4GVW5SSFIQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.216682051 (566h0m40.709114496s) [noise=true , address=145.79.227.19:4078 , piece_id=UDKV5GIBVIZGRFCC3N3BYVWYYVU6QX7N2VXSZINHI5RB6SA2ZG4A, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.218314572 (566h0m40.710747024s) [noise=true , address=45.91.116.7:4009 , piece_id=V35EMPA6QQ32EA55FNTLVBTTA6DH3ALHKJXS562TPFEVUVMQJTPQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.218755483 (566h0m40.711187936s) [noise=true , address=45.91.116.12:4227 , piece_id=OI2H2C33CY7YZXT3GB2IM3UOZEATOPUUAVK5AHFO2MVWNGK6OYTA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.219126261 (566h0m40.711558713s) [noise=true , address=45.91.116.11:4117 , piece_id=GUPR25YGE6U5SK34JIUAA5UPDLK3CUJPNLL64T63MM4W6CJ2HVJA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.219492564 (566h0m40.711925017s) [noise=true , address=77.90.31.14:4662 , piece_id=DOETS5GWB5TETOY2GU5BURWDBX64HWEEKDNFLQ2ENNPUSQXNTIOQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.219845801 (566h0m40.712278254s) [noise=true , address=145.79.227.11:4729 , piece_id=U7JH37SW7YHQP6LPYEVSS2BNPDCW3G4O4VWPL6FXDYBEMDKUTBHQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.220230214 (566h0m40.71266267s) [noise=true , address=45.91.116.9:4263 , piece_id=EXDHF74U2IBVZNY74S4M23CLFYNUUAKVDPWDUV6Z6Q3AEGUQJTMQ, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.221088298 (566h0m40.71352075s) [noise=true , address=145.79.227.2:4115 , piece_id=BL5OA2EFGTPOCOAR7RRHZR2SNYICU6N2XUH5GFVASJ3NJ7RIUX3A, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.221678245 (566h0m40.714110697s) [noise=true , address=77.90.31.19:4841 , piece_id=CT33BOQA5SAHHXD5H5VO4XXNAGJXOO3JI4SF6T3L5ZZ5DXN24NXA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.222178603 (566h0m40.714611054s) [noise=true , address=145.79.227.3:4151 , piece_id=XYBCZQEYZ77IG62ZQVVAUOPLGRCUKJQWBOCNQFHUE5MJ4MM33EBA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.222503950 (566h0m40.714936392s) [noise=true , address=145.79.227.5:4221 , piece_id=D5YEIOQRAH6VQRNWO3AXOFEF6EEQYA2LLPNDZ6TYQFJ2WH57PDIA, seg_pos={0 0}, upload=497944]: Uploading piece... Sep 22 04:08:20.843905532 (566h0m41.336337975s) [upload=497944]: Next returned nil segment Sep 22 04:08:20.843924538 (566h0m41.336356983s) [upload=497944]: Waiting for error group managing segments... Sep 22 04:08:20.974955345 (566h0m41.467387782s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZMW5KITYPRPR5GGQOY6HQOBDBDO2PUWETTMU7ZPYHVSRYMYPFWA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.976288091 (566h0m41.46872052s) [noise=true , address=45.91.116.4:4939 , piece_id=WL2H25EZ5M6LF4VIGCNOL7OTCG27CCY5VPER7IXPTQ4GVW5SSFIQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.976377050 (566h0m41.468809486s) [noise=true , address=45.91.116.7:4009 , piece_id=V35EMPA6QQ32EA55FNTLVBTTA6DH3ALHKJXS562TPFEVUVMQJTPQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.976677762 (566h0m41.46911019s) [noise=true , address=77.90.31.19:4841 , piece_id=CT33BOQA5SAHHXD5H5VO4XXNAGJXOO3JI4SF6T3L5ZZ5DXN24NXA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.979438006 (566h0m41.471870442s) [noise=true , address=45.91.116.9:4263 , piece_id=EXDHF74U2IBVZNY74S4M23CLFYNUUAKVDPWDUV6Z6Q3AEGUQJTMQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.980005232 (566h0m41.472437661s) [noise=true , address=45.91.116.12:4227 , piece_id=OI2H2C33CY7YZXT3GB2IM3UOZEATOPUUAVK5AHFO2MVWNGK6OYTA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.982100099 (566h0m41.474532534s) [noise=true , address=77.90.31.14:4662 , piece_id=DOETS5GWB5TETOY2GU5BURWDBX64HWEEKDNFLQ2ENNPUSQXNTIOQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.982843008 (566h0m41.475275436s) [noise=true , address=45.91.116.11:4117 , piece_id=GUPR25YGE6U5SK34JIUAA5UPDLK3CUJPNLL64T63MM4W6CJ2HVJA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:20.986232374 (566h0m41.478664805s) [noise=true , address=77.90.31.21:4878 , piece_id=7CUHFWHWCSFLGPXWBGMQL5IESETHDUBZ7RH35V6RUSL3O66RUSOQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: <nil> Sep 22 04:08:49.779601525 (566h1m10.272033958s) [noise=true , address=145.79.227.5:4221 , piece_id=D5YEIOQRAH6VQRNWO3AXOFEF6EEQYA2LLPNDZ6TYQFJ2WH57PDIA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: rpc: dial tcp 145.79.227.5:4221: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779670667 (566h1m10.272103099s) [noise=true , address=145.79.227.3:4151 , piece_id=XYBCZQEYZ77IG62ZQVVAUOPLGRCUKJQWBOCNQFHUE5MJ4MM33EBA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779695716 (566h1m10.272128149s) [noise=true , address=145.79.227.6:4259 , piece_id=SU6SZNPMK5X2MY7X3J4W4QUX32VVHSBLG52KTQMQWJVKGMNXKBYA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779851127 (566h1m10.272283563s) [noise=true , address=145.79.227.11:4729 , piece_id=U7JH37SW7YHQP6LPYEVSS2BNPDCW3G4O4VWPL6FXDYBEMDKUTBHQ, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779899437 (566h1m10.272331871s) [noise=true , address=145.79.227.19:4078 , piece_id=UDKV5GIBVIZGRFCC3N3BYVWYYVU6QX7N2VXSZINHI5RB6SA2ZG4A, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779910947 (566h1m10.272343382s) [noise=true , address=145.79.227.2:4115 , piece_id=BL5OA2EFGTPOCOAR7RRHZR2SNYICU6N2XUH5GFVASJ3NJ7RIUX3A, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779930053 (566h1m10.272362488s) [noise=true , address=145.79.227.13:4801 , piece_id=EXUBHS3HT4ENMTYZTEE3UT7CHPZ4BB5FA3LU6U7FTCMQZTAQULUA, seg_pos={0 0}, upload=497944]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:08:49.779987109 (566h1m10.272419559s) [seg_pos={0 0}, upload=497944]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 04:08:49.900199100 (566h1m10.392631537s) [upload=497944]: Done uploading segments... Sep 22 04:08:49.900247349 (566h1m10.392679799s) [upload=497944]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.12276728s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X...XX..........X.......X......X..XXXX....................] duration: 2.780640201s FAIL (2025-09-22 04:08:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.780667799s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.XX........X.X.X..................X..X..XX..X......XXX.....X] duration: 30.130017927s FAIL (2025-09-22 04:07:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :13.302313936 (565h59m33.794746382s) [upload=497929]: Starting upload Sep 22 04:07:13.302423245 (565h59m33.794855679s) [upload=497929]: Done starting upload Sep 22 04:07:13.302517771 (565h59m33.794950206s) [upload=497929]: Uploading segments... Sep 22 04:07:13.302879716 (565h59m33.795312159s) [upload=497929]: Got next segment. Inline: false Sep 22 04:07:13.302963275 (565h59m33.795395707s) [upload=497929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:07:13.302970930 (565h59m33.795403379s) [upload=497929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:07:13.422333202 (565h59m33.914765645s) [seg_pos={0 0}, upload=497929]: Begin upload segment... Sep 22 04:07:13.422416615 (565h59m33.914849045s) [seg_pos={0 0}, upload=497929]: Done begin upload segment. Sep 22 04:07:13.422443307 (565h59m33.914875759s) [noise=true , address=45.91.116.13:4154 , piece_id=V3PKF5ACPWIFPXHOPIYRTZ6MRGDP27C5MTH7LK4IJQ7KKNJVR7ZA, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.422450786 (565h59m33.914883228s) [noise=true , address=145.79.227.12:4767 , piece_id=A2S5WQIN53WCKTXNL7SPNMU2TCVVK3TJOS5BBQVDZKJU2G3KAWOA, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.422516693 (565h59m33.914949146s) [noise=true , address=145.79.227.5:4221 , piece_id=7NTZFHIZGFWXPK5DD4SYE3PEUYCFL5T4S6O2LRTJWMN74P5TGHJQ, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.422595646 (565h59m33.915028096s) [noise=true , address=45.91.116.7:4011 , piece_id=GIDNFAHALZTXNWEX54IL3WXVEPX4FI2QQVC4427X3LFFHBSZUIUA, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.423018395 (565h59m33.91545085s) [noise=true , address=145.79.227.2:4115 , piece_id=4OSH5NQH24N3WDNE2IC35GDASXRN2PKVHWKX2QVSHXR4B367EVKQ, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.423416896 (565h59m33.915849353s) [noise=true , address=145.79.227.13:4801 , piece_id=QMDUWKHHENPGX2LLIEETSQCY6HNCY3PH24EPVK6VU2NRMMYWHN4Q, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.423791506 (565h59m33.916223953s) [noise=true , address=77.90.31.12:4592 , piece_id=RJYDA3FZMUECYRY5M32KGOJBGESIXNVGTPZ72L5PXCNMT6HMR46A, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.424753832 (565h59m33.917186284s) [noise=true , address=77.90.31.19:4841 , piece_id=LLRCVAFQK4M3GY3XCO46PI6GOX4SGMWWCYPPJSFXHJBGWVB3VIXA, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.425695226 (565h59m33.918127678s) [noise=true , address=145.79.227.16:4911 , piece_id=LJD7K5AU6RISOUTNFKGHJS2633JW2YFMAYS7VXWDMQDUHYKESZCQ, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.427130611 (565h59m33.919563064s) [noise=true , address=145.79.227.4:4185 , piece_id=RQR6TN2VQZ5ZWWF66S2CP4UOMS6AQXKJG3MEDFJGTN6LXH7UTEBQ, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.428312548 (565h59m33.920745s) [noise=true , address=77.90.31.13:4626 , piece_id=6B5METBNLUCQLP3V4TIATYKBL7UJE6L3JPKG3YRSL3KK266S4QTQ, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.429633369 (565h59m33.922065864s) [noise=true , address=45.91.116.6:4974 , piece_id=FYIO3IV6F5TCV5OCRTNL3IRZCIRNDDBQPFUF72ROYPYLGHBQBH6A, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.430247010 (565h59m33.922679463s) [noise=true , address=145.79.227.19:4077 , piece_id=OM7O3JWFOX4WU45HRDIUNQTPG5CRCMG5PQ6S3DXUBOLCBZJSHVOA, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.430703067 (565h59m33.923135523s) [noise=true , address=45.91.116.4:4937 , piece_id=R3Z73O4FG5HJ2VLRP6Q4MUSHKMVIKEDZM3KHDVQVBTCVK5HIVU5A, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.431129273 (565h59m33.923561736s) [noise=true , address=45.91.116.9:4262 , piece_id=MMKJQLGSRJBEOTKBHL22MJ2F4TQWFZ3VQP5VURWXDMHQAKU7DIKA, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:13.432221393 (565h59m33.924653825s) [noise=true , address=145.79.227.14:4837 , piece_id=OANTRCSNM7DTARBUUOA2E32KIKSDDJV2SA5NS7NCTFM5JHZQFWDA, seg_pos={0 0}, upload=497929]: Uploading piece... Sep 22 04:07:14.036100096 (565h59m34.528532527s) [upload=497929]: Next returned nil segment Sep 22 04:07:14.036122322 (565h59m34.528554777s) [upload=497929]: Waiting for error group managing segments... Sep 22 04:07:14.173599393 (565h59m34.666031836s) [noise=true , address=45.91.116.13:4154 , piece_id=V3PKF5ACPWIFPXHOPIYRTZ6MRGDP27C5MTH7LK4IJQ7KKNJVR7ZA, seg_pos={0 0}, upload=497929]: Done uploading piece. err: <nil> Sep 22 04:07:14.174729383 (565h59m34.667161815s) [noise=true , address=77.90.31.12:4592 , piece_id=RJYDA3FZMUECYRY5M32KGOJBGESIXNVGTPZ72L5PXCNMT6HMR46A, seg_pos={0 0}, upload=497929]: Done uploading piece. err: <nil> Sep 22 04:07:14.178422463 (565h59m34.670854893s) [noise=true , address=77.90.31.13:4626 , piece_id=6B5METBNLUCQLP3V4TIATYKBL7UJE6L3JPKG3YRSL3KK266S4QTQ, seg_pos={0 0}, upload=497929]: Done uploading piece. err: <nil> Sep 22 04:07:14.182950035 (565h59m34.675382464s) [noise=true , address=77.90.31.19:4841 , piece_id=LLRCVAFQK4M3GY3XCO46PI6GOX4SGMWWCYPPJSFXHJBGWVB3VIXA, seg_pos={0 0}, upload=497929]: Done uploading piece. err: <nil> Sep 22 04:07:14.183135209 (565h59m34.675567656s) [noise=true , address=45.91.116.6:4974 , piece_id=FYIO3IV6F5TCV5OCRTNL3IRZCIRNDDBQPFUF72ROYPYLGHBQBH6A, seg_pos={0 0}, upload=497929]: Done uploading piece. err: <nil> Sep 22 04:07:14.184297268 (565h59m34.676729704s) [noise=true , address=45.91.116.7:4011 , piece_id=GIDNFAHALZTXNWEX54IL3WXVEPX4FI2QQVC4427X3LFFHBSZUIUA, seg_pos={0 0}, upload=497929]: Done uploading piece. err: <nil> Sep 22 04:07:14.187257518 (565h59m34.679689952s) [noise=true , address=45.91.116.9:4262 , piece_id=MMKJQLGSRJBEOTKBHL22MJ2F4TQWFZ3VQP5VURWXDMHQAKU7DIKA, seg_pos={0 0}, upload=497929]: Done uploading piece. err: <nil> Sep 22 04:07:14.188699913 (565h59m34.681132342s) [noise=true , address=45.91.116.4:4937 , piece_id=R3Z73O4FG5HJ2VLRP6Q4MUSHKMVIKEDZM3KHDVQVBTCVK5HIVU5A, seg_pos={0 0}, upload=497929]: Done uploading piece. err: <nil> Sep 22 04:07:43.002951006 (566h0m3.495383453s) [noise=true , address=145.79.227.4:4185 , piece_id=RQR6TN2VQZ5ZWWF66S2CP4UOMS6AQXKJG3MEDFJGTN6LXH7UTEBQ, seg_pos={0 0}, upload=497929]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:07:43.003031658 (566h0m3.495464093s) [noise=true , address=145.79.227.19:4077 , piece_id=OM7O3JWFOX4WU45HRDIUNQTPG5CRCMG5PQ6S3DXUBOLCBZJSHVOA, seg_pos={0 0}, upload=497929]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:07:43.003081805 (566h0m3.495514237s) [noise=true , address=145.79.227.2:4115 , piece_id=4OSH5NQH24N3WDNE2IC35GDASXRN2PKVHWKX2QVSHXR4B367EVKQ, seg_pos={0 0}, upload=497929]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:07:43.003453817 (566h0m3.495886249s) [noise=true , address=145.79.227.16:4911 , piece_id=LJD7K5AU6RISOUTNFKGHJS2633JW2YFMAYS7VXWDMQDUHYKESZCQ, seg_pos={0 0}, upload=497929]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:07:43.003498298 (566h0m3.49593073s) [noise=true , address=145.79.227.5:4221 , piece_id=7NTZFHIZGFWXPK5DD4SYE3PEUYCFL5T4S6O2LRTJWMN74P5TGHJQ, seg_pos={0 0}, upload=497929]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: rpc: dial tcp 145.79.227.5:4221: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:07:43.003541690 (566h0m3.495974123s) [noise=true , address=145.79.227.13:4801 , piece_id=QMDUWKHHENPGX2LLIEETSQCY6HNCY3PH24EPVK6VU2NRMMYWHN4Q, seg_pos={0 0}, upload=497929]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:07:43.003573609 (566h0m3.496006042s) [noise=true , address=145.79.227.12:4767 , piece_id=A2S5WQIN53WCKTXNL7SPNMU2TCVVK3TJOS5BBQVDZKJU2G3KAWOA, seg_pos={0 0}, upload=497929]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:07:43.003618292 (566h0m3.496050724s) [noise=true , address=145.79.227.14:4837 , piece_id=OANTRCSNM7DTARBUUOA2E32KIKSDDJV2SA5NS7NCTFM5JHZQFWDA, seg_pos={0 0}, upload=497929]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: rpc: dial tcp 145.79.227.14:4837: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:07:43.003634722 (566h0m3.49606717s) [seg_pos={0 0}, upload=497929]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Sep 22 04:07:43.130495044 (566h0m3.622927488s) [upload=497929]: Done uploading segments... Sep 22 04:07:43.130519632 (566h0m3.62295206s) [upload=497929]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.130115454s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...X...XX..........X.......X......X..XXXX...................] duration: 2.741953398s FAIL (2025-09-22 04:07:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.741974307s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X...XX..........X.......X......X..XXXX..................] duration: 2.683188065s FAIL (2025-09-22 04:06:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.683218461s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..XX........X.X.X..................X..X..XX..X......XXX.....] duration: 2.410864825s FAIL (2025-09-22 04:06:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:01:55.784336537 (565h54m16.276768987s) [upload=497843]: Starting upload Sep 22 04:01:55.784440717 (565h54m16.276873153s) [upload=497843]: Done starting upload Sep 22 04:01:55.784541542 (565h54m16.276973985s) [upload=497843]: Uploading segments... Sep 22 04:01:55.784846589 (565h54m16.277279033s) [upload=497843]: Got next segment. Inline: false Sep 22 04:01:55.784900518 (565h54m16.277332961s) [upload=497843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:01:55.784906939 (565h54m16.277339384s) [upload=497843]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:01:56.076586084 (565h54m16.569018538s) [seg_pos={0 0}, upload=497843]: Begin upload segment... Sep 22 04:01:56.076655128 (565h54m16.569087561s) [seg_pos={0 0}, upload=497843]: Done begin upload segment. Sep 22 04:01:56.076687864 (565h54m16.56912031s) [noise=true , address=77.90.31.21:4877 , piece_id=GH5BXNRR6YPYS5RFGDVWGGHFBT4BHE7Y5NYUKQ7FB4S3BJ3GKXZA, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.077228421 (565h54m16.569660865s) [noise=true , address=45.91.116.12:4226 , piece_id=GHHHP2DTHZTYTZVE7ROTL6VLGMYLAKMNXWG5XQQU7TZA7DYG37DQ, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.077291461 (565h54m16.569723903s) [noise=true , address=45.91.116.11:4119 , piece_id=5XMROXWAWAHDU3L446D2T667R4FT23F3MP4T5LUIW6PS6JUONV2Q, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.078154972 (565h54m16.570587414s) [noise=true , address=45.91.116.5:4297 , piece_id=KOAIBEMCOXAQ7USZO6IZ3QYDHGNH3UA5YTA5AKLEUFFFSEWMBGSA, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.078236633 (565h54m16.570669075s) [noise=true , address=145.79.227.15:4874 , piece_id=QLWSKQ2JUULOSWKC6YVGWMQL6ZLOWMKPRYSNASBX6R3FNULFFJQQ, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.078785736 (565h54m16.571218178s) [noise=true , address=145.79.227.6:4258 , piece_id=KBKCK7O5FWYVBPKJTBGUMPS7CM55IZA2EUWGCHIDDLEWRNJQRWCQ, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.078864851 (565h54m16.571297293s) [noise=true , address=45.91.116.13:4154 , piece_id=XYLC2C2S7CPKBGHC44AXOYKVD4CPGQMB5EQ424ZIRM4OPB2XLJJA, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.079399620 (565h54m16.571832065s) [noise=true , address=45.91.116.13:4153 , piece_id=GSBU5U4247JD7WCSCZTL27T4C5K7HPDL4NBDFROHNL77GRB53HYA, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.079464728 (565h54m16.571897173s) [noise=true , address=45.91.116.4:4937 , piece_id=DF5PITLREIKFM253QEMOTTH7QVJTVOSQXCKUPCYM2ZMZOV2JMMJA, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.080063448 (565h54m16.5724959s) [noise=true , address=45.91.116.6:4974 , piece_id=TZQCKGUIPIOC46VX2KOCZLVONRN3V7K4SE3A6K4HJEWL4QALYYUQ, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.080636764 (565h54m16.57306921s) [noise=true , address=145.79.227.13:4801 , piece_id=B23A3OLALHIDJG7V3CRIKFVZL6TMX2M3NTDIR5DZDCNM5XOOY27A, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.082277350 (565h54m16.574709795s) [noise=true , address=45.91.116.5:4298 , piece_id=VG5G2T5QS6NRN5AMYJMQ5DWXFKAQDUUQACECYLE4PIL3MWPOD5UQ, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.082621149 (565h54m16.575053596s) [noise=true , address=145.79.227.14:4839 , piece_id=VF5FRRETNNPFDULT4NMZ5HY4VJUZBO3K7XQD5NTNW7A6Z3QD7DFQ, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.082977454 (565h54m16.575409902s) [noise=true , address=77.90.31.14:4662 , piece_id=GRFFJOE2B4R4DVRWEYGBPSPS5HDM75D77B6PE2V6GIVIVJBHBSZA, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.084468691 (565h54m16.576901133s) [noise=true , address=77.90.31.10:4519 , piece_id=DOELXWFWS3YGGFMUVCCUE3KFA55I7JKM74GDHKMMKV7PG6NRXJAA, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.084700315 (565h54m16.577132765s) [noise=true , address=77.90.31.15:4699 , piece_id=I7JH6IMN2MLIHA4BASBQWZ6XUPJWUJGQ2B4CHZ6GK475O6STK3IA, seg_pos={0 0}, upload=497843]: Uploading piece... Sep 22 04:01:56.681785179 (565h54m17.174217632s) [upload=497843]: Next returned nil segment Sep 22 04:01:56.681807790 (565h54m17.174240242s) [upload=497843]: Waiting for error group managing segments... Sep 22 04:01:56.820157029 (565h54m17.312589464s) [noise=true , address=45.91.116.4:4937 , piece_id=DF5PITLREIKFM253QEMOTTH7QVJTVOSQXCKUPCYM2ZMZOV2JMMJA, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.822230925 (565h54m17.31466336s) [noise=true , address=45.91.116.5:4297 , piece_id=KOAIBEMCOXAQ7USZO6IZ3QYDHGNH3UA5YTA5AKLEUFFFSEWMBGSA, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.823056688 (565h54m17.315489122s) [noise=true , address=45.91.116.11:4119 , piece_id=5XMROXWAWAHDU3L446D2T667R4FT23F3MP4T5LUIW6PS6JUONV2Q, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.826436822 (565h54m17.318869257s) [noise=true , address=45.91.116.12:4226 , piece_id=GHHHP2DTHZTYTZVE7ROTL6VLGMYLAKMNXWG5XQQU7TZA7DYG37DQ, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.826534568 (565h54m17.318967015s) [noise=true , address=45.91.116.6:4974 , piece_id=TZQCKGUIPIOC46VX2KOCZLVONRN3V7K4SE3A6K4HJEWL4QALYYUQ, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.831116998 (565h54m17.323549431s) [noise=true , address=77.90.31.10:4519 , piece_id=DOELXWFWS3YGGFMUVCCUE3KFA55I7JKM74GDHKMMKV7PG6NRXJAA, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.836037950 (565h54m17.328470386s) [noise=true , address=45.91.116.5:4298 , piece_id=VG5G2T5QS6NRN5AMYJMQ5DWXFKAQDUUQACECYLE4PIL3MWPOD5UQ, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.843074793 (565h54m17.33550723s) [noise=true , address=77.90.31.15:4699 , piece_id=I7JH6IMN2MLIHA4BASBQWZ6XUPJWUJGQ2B4CHZ6GK475O6STK3IA, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.843294347 (565h54m17.335726796s) [noise=true , address=45.91.116.13:4154 , piece_id=XYLC2C2S7CPKBGHC44AXOYKVD4CPGQMB5EQ424ZIRM4OPB2XLJJA, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.849025288 (565h54m17.341457723s) [noise=true , address=45.91.116.13:4153 , piece_id=GSBU5U4247JD7WCSCZTL27T4C5K7HPDL4NBDFROHNL77GRB53HYA, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.860972622 (565h54m17.353405059s) [noise=true , address=77.90.31.21:4877 , piece_id=GH5BXNRR6YPYS5RFGDVWGGHFBT4BHE7Y5NYUKQ7FB4S3BJ3GKXZA, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:56.862753937 (565h54m17.35518637s) [noise=true , address=77.90.31.14:4662 , piece_id=GRFFJOE2B4R4DVRWEYGBPSPS5HDM75D77B6PE2V6GIVIVJBHBSZA, seg_pos={0 0}, upload=497843]: Done uploading piece. err: <nil> Sep 22 04:01:57.409633366 (565h54m17.902065793s) [noise=true , address=145.79.227.14:4839 , piece_id=VF5FRRETNNPFDULT4NMZ5HY4VJUZBO3K7XQD5NTNW7A6Z3QD7DFQ, seg_pos={0 0}, upload=497843]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: connect: no route to host; conn closed: op Sep 22 04:02:25.471520250 (565h54m45.963952691s) [noise=true , address=145.79.227.6:4258 , piece_id=KBKCK7O5FWYVBPKJTBGUMPS7CM55IZA2EUWGCHIDDLEWRNJQRWCQ, seg_pos={0 0}, upload=497843]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:02:25.471621694 (565h54m45.964054137s) [noise=true , address=145.79.227.13:4801 , piece_id=B23A3OLALHIDJG7V3CRIKFVZL6TMX2M3NTDIR5DZDCNM5XOOY27A, seg_pos={0 0}, upload=497843]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:02:25.471686261 (565h54m45.964118691s) [noise=true , address=145.79.227.15:4874 , piece_id=QLWSKQ2JUULOSWKC6YVGWMQL6ZLOWMKPRYSNASBX6R3FNULFFJQQ, seg_pos={0 0}, upload=497843]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: rpc: dial tcp 145.79.227.15:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:02:25.471709210 (565h54m45.964141658s) [seg_pos={0 0}, upload=497843]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:02:25.600024017 (565h54m46.092456446s) [upload=497843]: Done uploading segments... Sep 22 04:02:25.600060217 (565h54m46.092492663s) [upload=497843]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.410985358s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X...XX..........X.......X......X..XXXX.................] duration: 2.660358272s FAIL (2025-09-22 04:06:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.66041028s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)