Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.326927199s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:33:45 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.326963021s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.358817665s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:32:56 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.358910965s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.452539789s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:32:02 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.452571328s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.416520141s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:31:02 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.416568884s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.327017484s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:30:13 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.327045628s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.277681106s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:29:36 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.277714176s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.331629326s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:28:35 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.331668011s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 11.790050078s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:27:36 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 11.790180831s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.393972566s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:26:17 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.394047614s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.385776244s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:25:46 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.385801735s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.384796361s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:25:18 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.384834783s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.335091331s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:24:49 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.335117456s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.39519097s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:24:24 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.395246639s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.24373592s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:23:25 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.243760012s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.296317743s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:22:20 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.296348876s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.229902393s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:21:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.229938052s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.362780244s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:20:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.362808311s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.413782026s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:19:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.413822961s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.289674894s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:19:23 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.289706541s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.290602365s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:18:19 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.290628496s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.276206895s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:17:55 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.276238459s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.921328016s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:17:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.921358685s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.372397919s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:16:44 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.37242527s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.367416271s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:15:57 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.367446361s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.382028448s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:15:39 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.38205984s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.329157107s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:14:40 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.32918867s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.383086199s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:13:37 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.383117359s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.445301734s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:12:47 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.44533306s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.312286448s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:11:49 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.312330153s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.413934907s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:11:23 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.413966457s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.365054941s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:10:24 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.365085187s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.591490053s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:09:50 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.591623257s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.894004324s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:08:51 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.894037727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 4.691153727s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:07:47 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 4.691236784s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.271978446s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:07:07 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.27200846s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.683339288s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:06:42 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 5.6833602s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.433110522s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:06:22 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.433143733s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.3346026s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:05:44 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.334631647s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.356936041s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:05:14 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.356964892s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.236976338s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:04:58 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.2370013s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.382338897s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:04:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.382441881s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.339910854s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:03:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.339931876s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.365119154s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:03:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.365168261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.413272088s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:02:36 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.413328387s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.275123044s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:02:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.275158298s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.307991643s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:01:35 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.30802091s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.29791289s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:01:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.297947709s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.579322309s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 10:00:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.579375603s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.398247687s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 09:59:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.398298086s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.265163381s trace: https://jaeger.dev.storj.io/trace/2af447c1cc32e3fe FAIL (2025-11-11 09:58:46 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 11 09:56:38.891694755 (12m24.250353408s) [upload=174]: Starting upload Nov 11 09:56:38.891756957 (12m24.250415616s) [upload=174]: Done starting upload Nov 11 09:56:38.891883474 (12m24.250542118s) [upload=174]: Uploading segments... Nov 11 09:56:38.892161824 (12m24.250820469s) [upload=174]: Got next segment. Inline: false Nov 11 09:56:38.892646331 (12m24.251304991s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 11 09:56:38.892657501 (12m24.251316155s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 11 09:56:39.005052720 (12m24.363711398s) [seg_pos={0 0}, upload=174]: Begin upload segment... Nov 11 09:56:39.005125494 (12m24.363784134s) [seg_pos={0 0}, upload=174]: Done begin upload segment. Nov 11 09:56:39.005216129 (12m24.363874823s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005230740 (12m24.363889419s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.005999056 (12m24.364657738s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.006601206 (12m24.365259871s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007145644 (12m24.365804308s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.007818316 (12m24.366476979s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.008466282 (12m24.367124946s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.009115289 (12m24.367773953s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011091547 (12m24.36975021s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.011726540 (12m24.370385204s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.013122380 (12m24.371781047s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.015282432 (12m24.373941079s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.016746279 (12m24.375404942s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.019161006 (12m24.37781977s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.020917020 (12m24.379575685s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.023350963 (12m24.382009629s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Uploading piece... Nov 11 09:56:39.635609835 (12m24.994268492s) [upload=174]: Next returned nil segment Nov 11 09:56:39.635628781 (12m24.99428746s) [upload=174]: Waiting for error group managing segments... Nov 11 09:56:39.759546252 (12m25.118204907s) [noise=true , address=145.79.227.6:4258 , piece_id=ALGZJPRFE5ZKYDHSNSYWM437BF5SBGAQLV57FCSGFNRPDIO2XFTQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760153701 (12m25.118812356s) [noise=true , address=77.90.31.10:4519 , piece_id=U7FQQL65MZHQTWUOY5SKFL4VKT4QAFDVZ44EUOBKMNOLAADSSIPA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.760787126 (12m25.119445795s) [noise=true , address=145.79.227.13:4804 , piece_id=4YRWKCCB2UUB3TETJTW3DQLQACW33F3YIOOL4QCUHXCW4RFKRHHQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.761593519 (12m25.120252172s) [noise=true , address=145.79.227.27:4689 , piece_id=GF3HUNJX2JNDFNJMU643UNYV2NT7NRPRQ2CQNR3I6RZ2RLYCASJA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.762295832 (12m25.120954498s) [noise=true , address=145.79.227.11:4732 , piece_id=EAJQZUVBQDPB456TYONLDGEXU2T72DZIBCQQ3JX5BGHUTDGQD2IA, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766441496 (12m25.12510015s) [noise=true , address=145.79.227.28:4726 , piece_id=VQJAIUUVCF5UQTQI2I3S4JZWAABTNLRHHHVZ3YE6EZTTIZBCZ57A, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.766631748 (12m25.125290412s) [noise=true , address=145.79.227.32:4836 , piece_id=W5TK45TG3EV6AZTZN2UA5QHLQK6IXA46LVD2K5L6M6OXWQPRFN5Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767329514 (12m25.125988177s) [noise=true , address=77.90.31.17:4771 , piece_id=QEH6D7XZFBLANBZTAENBVN4LT6FPGTF6QC4XZ7T3BFD3BHIWQEOQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.767886254 (12m25.126544908s) [noise=true , address=145.79.227.9:4366 , piece_id=C4CZIP32EQME6H2GUZNJXQK4NM5GNOG7LIIGOKLOWWAQ2BKVHTVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.768539301 (12m25.12719796s) [noise=true , address=145.79.227.25:4619 , piece_id=FRKZ7ISY3ZD5TDIUXGFD6SUKNP6NQH6W2DP7ABT6AYWSCZGJUN7Q, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.787492893 (12m25.146151557s) [noise=true , address=45.91.116.6:4975 , piece_id=Z26AFA2WT4PQXOHPXC2LTHD6HGQW7KGNVFXYKAMLHCPIKVCQ7CWQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.789897656 (12m25.148556321s) [noise=true , address=145.79.227.21:4473 , piece_id=6JE3OOHCMUJ3HSR5BORCDCOXI7SFMIVKSMYOS3OLANHP6NIM6BVQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791424933 (12m25.150083592s) [noise=true , address=145.79.227.18:4006 , piece_id=ENXJAZBBU6PGPX56NFZQRLUGWKQ5CNP3LUMXIAMFKN5LBA3JO5PQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: <nil> Nov 11 09:56:39.791445282 (12m25.150103946s) [seg_pos={0 0}, upload=174]: Segment reached optimal threshold of 13 pieces. Nov 11 09:56:39.791713518 (12m25.150372181s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791725086 (12m25.150383768s) [noise=true , address=145.79.227.20:4438 , piece_id=RRWIZE2Y2XM3BX4KU5FH2HQYCJL3HAPBTJXBBCCD2FUQT5DHVKPQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.791778521 (12m25.150437247s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.791793004 (12m25.150451669s) [noise=true , address=45.91.116.9:4263 , piece_id=HQKC6IELCZFPVCWI36TR63KYHI3Y35BLU3SMVWCR37ETJWZBAA3A, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792081601 (12m25.150740263s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AbLS96MKBYqYykQQJbAFHhfcmturggwPPCsNZ4oGzyMNugVfG): protocol: expected piece hash; context canceled; EOF Nov 11 09:56:39.792090492 (12m25.150749206s) [noise=true , address=145.79.227.22:4511 , piece_id=KI5VRUSEKKHQ2HPTZW3MSOZZSSVITYTG4GPUZMLFJ4ZXI7UOC4CQ, seg_pos={0 0}, upload=174]: Long tail cancellation due to optimal threshold Nov 11 09:56:39.792110632 (12m25.150769284s) [seg_pos={0 0}, upload=174]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 11 09:56:39.792218914 (12m25.150877558s) [upload=174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 11 09:56:39.792224633 (12m25.150883277s) [upload=174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 11 09:56:39.931652725 (12m25.29031138s) [upload=174]: Done uploading segments... Nov 11 09:56:39.931673903 (12m25.290332558s) [upload=174]: Upload finished. err: <nil> Duration: 2.265192854s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)