Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001251291s FAIL (2025-06-26 06:39:41 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 30.001276229s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 2.269442003s FAIL (2025-06-26 06:39:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.269477704s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 2.77117179s FAIL (2025-06-26 06:39:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.771985427s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 2.337709778s FAIL (2025-06-26 06:38:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.337743463s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 2.576611391s FAIL (2025-06-26 06:38:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.576637513s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 2.349728656s FAIL (2025-06-26 06:37:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.34979742s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 2.813372197s FAIL (2025-06-26 06:37:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.813393604s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 2.828274281s FAIL (2025-06-26 06:37:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.828336924s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 2.352836355s FAIL (2025-06-26 06:36:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.352906881s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 2.349343575s FAIL (2025-06-26 06:36:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.349375783s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 2.792128113s FAIL (2025-06-26 06:36:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.792155964s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 2.85143398s FAIL (2025-06-26 06:36:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.851474354s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 2.974700389s FAIL (2025-06-26 06:35:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.974722608s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 2.282295317s FAIL (2025-06-26 06:35:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.282346562s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 2.550565812s FAIL (2025-06-26 06:35:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.550595585s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 2.706150695s FAIL (2025-06-26 06:35:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.706211789s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 2.693391145s FAIL (2025-06-26 06:34:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.693415663s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 2.678978462s FAIL (2025-06-26 06:34:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.679006402s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 2.678403288s FAIL (2025-06-26 06:33:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.67842984s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 2.378377353s FAIL (2025-06-26 06:33:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.378421994s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 2.718645827s FAIL (2025-06-26 06:33:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.718683908s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 2.276462451s FAIL (2025-06-26 06:32:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.27649684s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 2.635134483s FAIL (2025-06-26 06:32:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.635221669s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 2.318571401s FAIL (2025-06-26 06:31:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.31860072s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 2.876577287s FAIL (2025-06-26 06:31:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.876606973s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 2.326486372s FAIL (2025-06-26 06:31:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.326514603s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 2.363570747s FAIL (2025-06-26 06:30:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.363607079s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 2.68746897s FAIL (2025-06-26 06:30:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.687519938s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 2.709736591s FAIL (2025-06-26 06:30:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.70976258s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 2.267281021s FAIL (2025-06-26 06:29:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.267315249s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 2.741285671s FAIL (2025-06-26 06:29:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.741353424s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 2.580062336s FAIL (2025-06-26 06:28:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.580089237s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 2.442253083s FAIL (2025-06-26 06:28:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.442287419s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 2.538969608s FAIL (2025-06-26 06:28:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.539025405s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 2.795622662s FAIL (2025-06-26 06:27:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.795657245s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 2.346001753s FAIL (2025-06-26 06:27:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.34602857s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 2.27712943s FAIL (2025-06-26 06:27:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.277169127s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 2.660591559s FAIL (2025-06-26 06:27:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.660613387s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 2.694516886s FAIL (2025-06-26 06:26:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.694555547s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 2.507289261s FAIL (2025-06-26 06:26:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.507384433s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 2.843667585s FAIL (2025-06-26 06:25:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.843743687s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 2.964300998s FAIL (2025-06-26 06:25:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.964339772s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 2.59653807s FAIL (2025-06-26 06:24:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.596562527s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 2.324524021s FAIL (2025-06-26 06:24:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.324565199s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 2.623941319s FAIL (2025-06-26 06:24:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.62397541s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 2.308225854s FAIL (2025-06-26 06:23:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.308254601s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 2.562387881s FAIL (2025-06-26 06:23:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.562472568s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 2.383416334s FAIL (2025-06-26 06:23:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.383468954s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 2.773297836s FAIL (2025-06-26 06:22:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.773351542s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 2.39227019s FAIL (2025-06-26 06:22:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ed. err: failed to upload enough pieces (needed at least 13 but got 11) Jun 26 06:08:46.149410115 (68h57m46.032823751s) [upload=55164]: Starting upload Jun 26 06:08:46.149511905 (68h57m46.032925531s) [upload=55164]: Done starting upload Jun 26 06:08:46.149795315 (68h57m46.033208934s) [upload=55164]: Uploading segments... Jun 26 06:08:46.149856673 (68h57m46.033270291s) [upload=55164]: Got next segment. Inline: false Jun 26 06:08:46.149949372 (68h57m46.033362999s) [upload=55164]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 26 06:08:46.149955991 (68h57m46.033369622s) [upload=55164]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 26 06:08:46.256737978 (68h57m46.140151591s) [seg_pos={0 0}, upload=55164]: Begin upload segment... Jun 26 06:08:46.256791847 (68h57m46.140205461s) [seg_pos={0 0}, upload=55164]: Done begin upload segment. Jun 26 06:08:46.256817169 (68h57m46.140230797s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.256845089 (68h57m46.140258718s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257153939 (68h57m46.140567569s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257402464 (68h57m46.140816091s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.257702039 (68h57m46.141115666s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.258771691 (68h57m46.14218533s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.259903667 (68h57m46.143317298s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260178371 (68h57m46.143591997s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260466182 (68h57m46.143879836s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.260753254 (68h57m46.144166867s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261021742 (68h57m46.144435369s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261275360 (68h57m46.144688989s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.261526496 (68h57m46.144940122s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262053155 (68h57m46.14546678s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262502914 (68h57m46.145916543s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.262903705 (68h57m46.146317334s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Uploading piece... Jun 26 06:08:46.864576292 (68h57m46.74798991s) [upload=55164]: Next returned nil segment Jun 26 06:08:46.864595177 (68h57m46.748008793s) [upload=55164]: Waiting for error group managing segments... Jun 26 06:08:47.003788840 (68h57m46.88720247s) [noise=true , address=104.36.82.16:4294 , piece_id=UJZR7PVKMQHLVCG7LYJP6X4XO2UIV2V36WLGQ6G5VZNUDKN5GRTQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.005691175 (68h57m46.889104787s) [noise=true , address=104.36.82.13:4187 , piece_id=7TTSKMLMBKTCMVCDZF4IF5LUREE625KNTNT7B2DT7L2UIHPBU3TQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.007200494 (68h57m46.890614107s) [noise=true , address=104.36.82.11:4115 , piece_id=PMIRDCZITRUNULQCVTHC5EMJRJNYAVCUAXBTWVGMIHPPSGVLTPLA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008311260 (68h57m46.891724874s) [noise=true , address=104.36.82.13:4185 , piece_id=Y4PKYNZGTJE7AARTXY6VDXHHWEEMV2JZEQZWBPHG2PM26EIUD7CQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.008467994 (68h57m46.891881608s) [noise=true , address=104.36.82.2:4729 , piece_id=RQ4OP57NLIB2ANTMIHJXVGOYGC6MW53D3CTJJH6ES7MT5KPCY4SA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.009777908 (68h57m46.893191536s) [noise=true , address=104.36.82.10:4078 , piece_id=4LZBTFKJSUJMZLQARME6BXVB3H5WESFZNW25YJ2R6GHZT2OXL24A, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.043010535 (68h57m46.92642415s) [noise=true , address=104.36.82.12:4151 , piece_id=WOKRYNB4PN7ECNIBJE3NFGHFAGWI76RFRWZZVI2INAEK6WDKIGDA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.047598840 (68h57m46.931012476s) [noise=true , address=104.36.82.7:4912 , piece_id=BU4CEFCBKQT7AWQZUAZPOHSVMUXNNDKQTSC7C56XRVRWNNP6DO7Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:08:47.193461295 (68h57m47.076874929s) [noise=true , address=104.36.82.5:4839 , piece_id=UWT4YOSX56AFHVPNI2QCMZBWQBE5DR4744U4F626VBGLYWE5MTWA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: <nil> Jun 26 06:09:15.876852535 (68h58m15.760266169s) [noise=true , address=45.91.116.7:4009 , piece_id=HBHKYHIZEVAZOMQZSMXAOHVRS2AZTUKKCBOZSE6BKICLBD3YX4MQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.876933547 (68h58m15.760347166s) [noise=true , address=45.91.116.4:4939 , piece_id=6772RQD2SQSIXK5Y5MIC3TI3YLPF7YHK4HU2C46S7ZGTDFW2A2QQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:42128->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877005614 (68h58m15.760419242s) [noise=true , address=77.90.31.15:4699 , piece_id=QGFTD3CAGNOSOERNCSQJY2HSQ32QPJYLCDM6EZ3J46427YMAPOJA, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 139.59.144.22:34760->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877091107 (68h58m15.760504723s) [noise=true , address=45.91.116.5:5298 , piece_id=42RTBIB7AC4ESAFENJA42VCI53PZ6IYXFF7YWXCCCEOHPTYKHQ6Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:5298): protocol: write tcp 139.59.144.22:38610->45.91.116.5:5298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877252123 (68h58m15.760665755s) [noise=true , address=45.91.116.10:4083 , piece_id=QAS5LP63LNLVPTV2HXP2ISP5GOF6434XIWBWAJDD45SZYXY6RE3Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 139.59.144.22:39986->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877335452 (68h58m15.760749085s) [noise=true , address=77.90.31.17:4772 , piece_id=CVT4UOXEQUFIJFISCYFLZFMN5TZTULOINTVGRWZP3KK5WFCJRYCQ, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 139.59.144.22:57926->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877412512 (68h58m15.760826142s) [noise=true , address=77.90.31.12:4590 , piece_id=PU3VK4OCFSSUP7SQOD2EF2FN2F24OZZM5IJ45XFNTKDI5ZDLJT2Q, seg_pos={0 0}, upload=55164]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 139.59.144.22:39360->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jun 26 06:09:15.877469294 (68h58m15.760882924s) [seg_pos={0 0}, upload=55164]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jun 26 06:09:15.983403909 (68h58m15.866817535s) [upload=55164]: Done uploading segments... Jun 26 06:09:15.983433081 (68h58m15.866846707s) [upload=55164]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.392305098s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)