Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 2.293306278s FAIL (2025-06-26 06:54:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.293335453s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 2.29517928s FAIL (2025-06-26 06:54:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.295207399s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.212357151s FAIL (2025-06-26 06:54:04 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.212396433s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 2.277930886s FAIL (2025-06-26 06:53:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.277981386s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.314161268s FAIL (2025-06-26 06:53:19 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.314193494s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 2.295474854s FAIL (2025-06-26 06:53:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.295509276s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.161699977s FAIL (2025-06-26 06:52:40 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.161732968s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.071401824s FAIL (2025-06-26 06:52:19 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.071435152s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 2.27145484s FAIL (2025-06-26 06:52:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.271585809s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.243933338s FAIL (2025-06-26 06:51:49 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.244004757s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 2.315733256s FAIL (2025-06-26 06:50:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.315764707s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.548171927s FAIL (2025-06-26 06:50:49 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.548349393s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.796765249s FAIL (2025-06-26 06:50:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.796796487s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 2.304328757s FAIL (2025-06-26 06:49:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.304362677s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.573098411s FAIL (2025-06-26 06:49:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.573126925s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.199196198s FAIL (2025-06-26 06:49:37 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.199231395s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 2.813833651s FAIL (2025-06-26 06:49:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.813854347s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 2.280029909s FAIL (2025-06-26 06:49:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.28006565s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 2.672459366s FAIL (2025-06-26 06:48:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.672487411s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.068845966s FAIL (2025-06-26 06:48:33 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.068882107s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 2.31415772s FAIL (2025-06-26 06:48:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.314205315s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 2.613949227s FAIL (2025-06-26 06:47:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.613976829s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 2.294161612s FAIL (2025-06-26 06:47:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.294195939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.522531251s FAIL (2025-06-26 06:47:33 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.522569643s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.98867885s FAIL (2025-06-26 06:47:13 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 1.988716389s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 2.742803458s FAIL (2025-06-26 06:46:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.742887574s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.076450788s FAIL (2025-06-26 06:46:56 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.076536744s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 2.404394455s FAIL (2025-06-26 06:46:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.404455936s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.102803243s FAIL (2025-06-26 06:46:37 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.102834543s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.143342326s FAIL (2025-06-26 06:46:17 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.143379585s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 2.679849332s FAIL (2025-06-26 06:45:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.679879735s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 2.348202534s FAIL (2025-06-26 06:45:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.348241259s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.168246626s FAIL (2025-06-26 06:45:32 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.168329549s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.68055365s FAIL (2025-06-26 06:45:04 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.680646174s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 3.025560228s FAIL (2025-06-26 06:45:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.025591495s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.164751696s FAIL (2025-06-26 06:44:42 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.164784033s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 2.33774817s FAIL (2025-06-26 06:44:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.337780224s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 2.631687373s FAIL (2025-06-26 06:44:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.631713887s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 2.847418307s FAIL (2025-06-26 06:44:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.84743919s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.259095972s FAIL (2025-06-26 06:44:09 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.259127702s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.44499724s FAIL (2025-06-26 06:43:44 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.445036773s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 2.363164295s FAIL (2025-06-26 06:43:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.363213129s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 2.772788214s FAIL (2025-06-26 06:43:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.772815372s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 2.345236436s FAIL (2025-06-26 06:43:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.34526715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.540350039s FAIL (2025-06-26 06:42:55 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.540439672s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 2.949747707s FAIL (2025-06-26 06:42:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.949773883s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.000889596s FAIL (2025-06-26 06:42:14 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 2.000919266s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 2.320824977s FAIL (2025-06-26 06:42:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.320858902s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 2.258608749s FAIL (2025-06-26 06:41:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.258643787s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 2.516690986s FAIL (2025-06-26 06:41:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.516725664s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)