Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X..................X] duration: 1m0.123960502s FAIL (2025-09-03 20:35:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== SegmentBegin Sep 3 20:34:45.539874378 (126h27m6.032306821s) [seg_pos={0 0}, upload=111337]: Begin upload segment... Sep 3 20:34:45.539944354 (126h27m6.032376798s) [seg_pos={0 0}, upload=111337]: Done begin upload segment. Sep 3 20:34:45.539984380 (126h27m6.032416826s) [noise=true , address=104.36.82.55:4217 , piece_id=EDTM4MWNLF6VHOB5PD5WMKXQSDOOLTG3Q2NHOFHCRAAD7MCCPA2A, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.539993246 (126h27m6.032425688s) [noise=true , address=45.91.116.11:4151 , piece_id=L2MLKLHBLT7TDLFCVBR7RVZYMV2FSJ7B5PGXFRGYESM3VBYP5FUA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.540504660 (126h27m6.032937104s) [noise=true , address=77.90.31.14:4677 , piece_id=64AIIETVLCBZX2KC7DKC63D3XTQDTYBTGOVMA2RPY4UJPUNORQAA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.540897445 (126h27m6.03332989s) [noise=true , address=104.36.82.60:4364 , piece_id=OTRNWCHMJKCF4D2CEMWNDAMM6IO2Z6EZDG5TTJQABCMEOS3NAROQ, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.541256572 (126h27m6.033689015s) [noise=true , address=45.91.116.10:4101 , piece_id=VJHCCVNFJW63KXRVPOKJ6UOXKER6KDXWSQE7N43QYWT2D6D4XAVA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.541627394 (126h27m6.034059838s) [noise=true , address=104.36.82.51:4076 , piece_id=JXUMTJIZQA2TIOWCGWTJPBXYVF3SK2R5ATUF6DJDJT6WCHIMGPSQ, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.542792735 (126h27m6.03522519s) [noise=true , address=104.36.82.5:4864 , piece_id=EZWEBNB235O76W2OR6MNWEC2AOBSPDB4DOG7EYSPCMWLR3SVPJLA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.543504785 (126h27m6.035937242s) [noise=true , address=104.36.82.56:4236 , piece_id=HY6LOBSUJQW3CQKTIMEO6L5PNSFNDORXRDJXGVMOE6EURHZGQ7VA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.544083119 (126h27m6.036515567s) [noise=true , address=104.36.82.62:4429 , piece_id=34DEHHLY3E4CSNUSMIAGEIXGVO2Z3ZYIZDQGKOTFZT6UNBEUDBQQ, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.544634763 (126h27m6.037067214s) [noise=true , address=104.36.82.14:4250 , piece_id=MK7AM3LZNZGHF5PJ2WAX4KU7TKJDQY5SZOBTWEIIWHSBOPJDHIMQ, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.545179826 (126h27m6.037612279s) [noise=true , address=104.36.82.52:4109 , piece_id=SOADOSKWVYMRZVKD7BLLLLKG3UZXTWARLS2EA7T6FIYMY2SGOFGA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.545729148 (126h27m6.0381616s) [noise=true , address=104.36.82.4:4823 , piece_id=2EQUWNHGBXUUYEBCCJE5BW2HXVONGDLZVSTFW25B6Z477KH3YRSA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.546368408 (126h27m6.038800859s) [noise=true , address=77.90.31.19:4848 , piece_id=YGHRY2QU4RPI6FXVSHUPAXLHXZXCJYRSOG2NXTHR5TEFDGHXQFIQ, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.546939915 (126h27m6.039372362s) [noise=true , address=104.36.82.23:4477 , piece_id=5AJRTF4G7YJBOPVZ54PPFQWHL4BNZHSTBVZOXJPSGM5VJLJFH22A, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.547395598 (126h27m6.039828046s) [noise=true , address=64.49.13.155:4918 , piece_id=RPVG6OJKZA4RNUFQSGVYHRU5GXR3NTYB2T77KOUCFQ33YH3JH6LA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.547727168 (126h27m6.040159616s) [noise=true , address=77.90.31.19:4872 , piece_id=LYVDAURQJZFULEEOQCZC4AKNBWS54LUMZCBRM24QPLJFZX5YDUQA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.548026339 (126h27m6.04045878s) [noise=true , address=104.36.82.61:4392 , piece_id=RYET4ZHKCUP7INJUINY52PJJVS5WEV57ORQZVB3ABX4HRJV3OXOA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:45.548375197 (126h27m6.040807633s) [noise=true , address=64.49.13.138:4488 , piece_id=4KT2EDSVGC2GPEKM5TFXIGTGPJRBK7GDUDVCDDG6ETDNXZXL3RCA, seg_pos={0 0}, upload=111337]: Uploading piece... Sep 3 20:34:46.100594181 (126h27m6.593026623s) [upload=111337]: Next returned nil segment Sep 3 20:34:46.100612698 (126h27m6.593045131s) [upload=111337]: Waiting for error group managing segments... Sep 3 20:34:46.279189059 (126h27m6.771621488s) [noise=true , address=45.91.116.10:4101 , piece_id=VJHCCVNFJW63KXRVPOKJ6UOXKER6KDXWSQE7N43QYWT2D6D4XAVA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.300053061 (126h27m6.792485497s) [noise=true , address=77.90.31.19:4848 , piece_id=YGHRY2QU4RPI6FXVSHUPAXLHXZXCJYRSOG2NXTHR5TEFDGHXQFIQ, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.314839853 (126h27m6.807272289s) [noise=true , address=77.90.31.19:4872 , piece_id=LYVDAURQJZFULEEOQCZC4AKNBWS54LUMZCBRM24QPLJFZX5YDUQA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.383653580 (126h27m6.876086029s) [noise=true , address=45.91.116.11:4151 , piece_id=L2MLKLHBLT7TDLFCVBR7RVZYMV2FSJ7B5PGXFRGYESM3VBYP5FUA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.645381914 (126h27m7.137814351s) [noise=true , address=104.36.82.60:4364 , piece_id=OTRNWCHMJKCF4D2CEMWNDAMM6IO2Z6EZDG5TTJQABCMEOS3NAROQ, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.784953418 (126h27m7.277385862s) [noise=true , address=104.36.82.56:4236 , piece_id=HY6LOBSUJQW3CQKTIMEO6L5PNSFNDORXRDJXGVMOE6EURHZGQ7VA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.798857895 (126h27m7.291290338s) [noise=true , address=64.49.13.155:4918 , piece_id=RPVG6OJKZA4RNUFQSGVYHRU5GXR3NTYB2T77KOUCFQ33YH3JH6LA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.820576240 (126h27m7.313008673s) [noise=true , address=64.49.13.138:4488 , piece_id=4KT2EDSVGC2GPEKM5TFXIGTGPJRBK7GDUDVCDDG6ETDNXZXL3RCA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.878635117 (126h27m7.371067559s) [noise=true , address=104.36.82.61:4392 , piece_id=RYET4ZHKCUP7INJUINY52PJJVS5WEV57ORQZVB3ABX4HRJV3OXOA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:46.973607379 (126h27m7.466039808s) [noise=true , address=104.36.82.62:4429 , piece_id=34DEHHLY3E4CSNUSMIAGEIXGVO2Z3ZYIZDQGKOTFZT6UNBEUDBQQ, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:47.636395704 (126h27m8.12882816s) [noise=true , address=104.36.82.23:4477 , piece_id=5AJRTF4G7YJBOPVZ54PPFQWHL4BNZHSTBVZOXJPSGM5VJLJFH22A, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:34:53.270676163 (126h27m13.763108591s) [noise=true , address=77.90.31.14:4677 , piece_id=64AIIETVLCBZX2KC7DKC63D3XTQDTYBTGOVMA2RPY4UJPUNORQAA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: <nil> Sep 3 20:35:44.977585465 (126h28m5.470017909s) [noise=true , address=104.36.82.51:4076 , piece_id=JXUMTJIZQA2TIOWCGWTJPBXYVF3SK2R5ATUF6DJDJT6WCHIMGPSQ, seg_pos={0 0}, upload=111337]: Done uploading piece. err: ecclient: upload failed (node:19KofWeQV4jMQzmSCv3LUsUqAqGfAuDqyo9GGsqKGi7ctT3FA2, address:104.36.82.51:4076): protocol: write tcp 139.59.144.22:56878->104.36.82.51:4076: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:35:44.977661687 (126h28m5.470094115s) [noise=true , address=104.36.82.55:4217 , piece_id=EDTM4MWNLF6VHOB5PD5WMKXQSDOOLTG3Q2NHOFHCRAAD7MCCPA2A, seg_pos={0 0}, upload=111337]: Done uploading piece. err: ecclient: upload failed (node:12bw9V8UMi7pFxqJ8fh22x5ZPmtJUJyJGw8hD9NhQcfHdfx4cZj, address:104.36.82.55:4217): protocol: write tcp 139.59.144.22:59252->104.36.82.55:4217: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:35:44.977798951 (126h28m5.47023138s) [noise=true , address=104.36.82.5:4864 , piece_id=EZWEBNB235O76W2OR6MNWEC2AOBSPDB4DOG7EYSPCMWLR3SVPJLA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: ecclient: upload failed (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u, address:104.36.82.5:4864): protocol: write tcp 139.59.144.22:51374->104.36.82.5:4864: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:35:44.977885569 (126h28m5.470317998s) [noise=true , address=104.36.82.4:4823 , piece_id=2EQUWNHGBXUUYEBCCJE5BW2HXVONGDLZVSTFW25B6Z477KH3YRSA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: ecclient: upload failed (node:1JWhB5QxbZsTD863tEfkEN1CUbKBMQvMJ9ZaiMAwKvxxXyDkEq, address:104.36.82.4:4823): protocol: rpc: dial tcp 104.36.82.4:4823: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:35:44.977895920 (126h28m5.470328349s) [noise=true , address=104.36.82.52:4109 , piece_id=SOADOSKWVYMRZVKD7BLLLLKG3UZXTWARLS2EA7T6FIYMY2SGOFGA, seg_pos={0 0}, upload=111337]: Done uploading piece. err: ecclient: upload failed (node:1hBqAk4HrRmUKRw3RS7xUHZ4rdUMiF25jKWF7jqNrffZG8pEcj, address:104.36.82.52:4109): protocol: write tcp 139.59.144.22:58910->104.36.82.52:4109: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:35:44.977980275 (126h28m5.470412704s) [noise=true , address=104.36.82.14:4250 , piece_id=MK7AM3LZNZGHF5PJ2WAX4KU7TKJDQY5SZOBTWEIIWHSBOPJDHIMQ, seg_pos={0 0}, upload=111337]: Done uploading piece. err: ecclient: upload failed (node:1uoGjfqBLfoNrg2X1aYKYLKQjgFUnzLjHetCFc9sPjJa2qqKYM, address:104.36.82.14:4250): protocol: write tcp 139.59.144.22:47576->104.36.82.14:4250: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:35:44.978025216 (126h28m5.470457661s) [seg_pos={0 0}, upload=111337]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:35:45.099009729 (126h28m5.591442165s) [upload=111337]: Done uploading segments... Sep 3 20:35:45.099039156 (126h28m5.591471592s) [upload=111337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.124013964s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX.....X..X......XX......X......X....X......................] duration: 2.547773743s FAIL (2025-09-03 20:35:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.547812278s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX.............] duration: 2.811567757s FAIL (2025-09-03 20:34:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.811744273s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXX.....X..X......XX......X......X....X.....................] duration: 3.24773146s FAIL (2025-09-03 20:34:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 3.247966205s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX............] duration: 2.847556444s FAIL (2025-09-03 20:34:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.847584849s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXX.....X..X......XX......X......X....X....................] duration: 2.557498547s FAIL (2025-09-03 20:33:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.557598922s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX...........] duration: 2.806124299s FAIL (2025-09-03 20:33:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.806150863s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.984614567s FAIL (2025-09-03 20:33:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 2.984645255s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [XXXXX.....X..X......XX......X......X....X...................] duration: 2.415446125s FAIL (2025-09-03 20:33:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.41547805s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX..........] duration: 2.937249004s FAIL (2025-09-03 20:33:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.937275154s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 3.55336779s FAIL (2025-09-03 20:32:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.55346622s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.XXXXX.....X..X......XX......X......X....X..................] duration: 5.525149574s FAIL (2025-09-03 20:32:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.525182301s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX.........] duration: 2.752719175s FAIL (2025-09-03 20:32:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.752754404s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [...X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX........] duration: 3.029230886s FAIL (2025-09-03 20:32:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.029259918s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX.......] duration: 2.860530884s FAIL (2025-09-03 20:32:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.860600301s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X.XXXXX.....X..X......XX......X......X....X.................] duration: 2.524211495s FAIL (2025-09-03 20:31:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.524240773s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 4.458332542s FAIL (2025-09-03 20:31:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 4.458361023s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX......] duration: 3.388385662s FAIL (2025-09-03 20:31:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.38848079s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 3.341924412s FAIL (2025-09-03 20:31:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.341979317s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.208873302s FAIL (2025-09-03 20:30:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 5.208903601s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X.XXXXX.....X..X......XX......X......X....X................] duration: 7.772294919s FAIL (2025-09-03 20:30:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.772330854s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 3.180225238s FAIL (2025-09-03 20:30:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.180264314s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX.....] duration: 2.847136881s FAIL (2025-09-03 20:30:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.847381688s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 3.966504533s FAIL (2025-09-03 20:30:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.966541098s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X.X.XXXXX.....X..X......XX......X......X....X...............] duration: 2.596544211s FAIL (2025-09-03 20:29:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.596580691s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [..X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX....] duration: 2.682193622s FAIL (2025-09-03 20:29:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.682218356s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.560106781s FAIL (2025-09-03 20:29:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 2.56013551s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [XX.X.XXXXX.....X..X......XX......X......X....X..............] duration: 2.426416401s FAIL (2025-09-03 20:29:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.42646301s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [XXX.X.XXXXX.....X..X......XX......X......X....X.............] duration: 2.655197679s FAIL (2025-09-03 20:29:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.655281565s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.556320991s FAIL (2025-09-03 20:28:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 2.556356869s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.XXX.X.XXXXX.....X..X......XX......X......X....X............] duration: 2.34546491s FAIL (2025-09-03 20:28:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.345490795s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [...X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX...] duration: 2.843107176s FAIL (2025-09-03 20:28:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.843607969s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.64301844s FAIL (2025-09-03 20:28:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 2.643049941s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..XXX.X.XXXXX.....X..X......XX......X......X....X...........] duration: 2.406884329s FAIL (2025-09-03 20:28:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.406917168s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 3.181959249s FAIL (2025-09-03 20:27:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.181989106s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [...XXX.X.XXXXX.....X..X......XX......X......X....X..........] duration: 2.660340569s FAIL (2025-09-03 20:27:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.66037067s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [....X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX..] duration: 2.920526694s FAIL (2025-09-03 20:27:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.920565711s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 3.488242915s FAIL (2025-09-03 20:27:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.488281485s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 3.114973315s FAIL (2025-09-03 20:26:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.115036662s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.....X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX.] duration: 2.826543439s FAIL (2025-09-03 20:26:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.826562569s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [....XXX.X.XXXXX.....X..X......XX......X......X....X.........] duration: 2.850584959s FAIL (2025-09-03 20:26:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.850615196s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [......X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........XX] duration: 30.000465354s FAIL (2025-09-03 20:26:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000481189s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.......X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........X] duration: 30.001089819s FAIL (2025-09-03 20:25:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001115019s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.....XXX.X.XXXXX.....X..X......XX......X......X....X........] duration: 2.708393954s FAIL (2025-09-03 20:25:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.708427145s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 3.183219883s FAIL (2025-09-03 20:25:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.183249646s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 3.214417225s FAIL (2025-09-03 20:25:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.214450791s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [......XXX.X.XXXXX.....X..X......XX......X......X....X.......] duration: 2.576920145s FAIL (2025-09-03 20:25:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.576969242s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.624696425s FAIL (2025-09-03 20:24:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== d Sep 3 20:22:37.745083823 (126h14m58.237516266s) [upload=111152]: Uploading segments... Sep 3 20:22:37.745206727 (126h14m58.23763917s) [upload=111152]: Got next segment. Inline: false Sep 3 20:22:37.745290193 (126h14m58.237722644s) [upload=111152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:22:37.745311970 (126h14m58.237744415s) [upload=111152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:22:37.978073728 (126h14m58.470506185s) [seg_pos={0 0}, upload=111152]: Begin upload segment... Sep 3 20:22:37.978158307 (126h14m58.470590738s) [seg_pos={0 0}, upload=111152]: Done begin upload segment. Sep 3 20:22:37.978183237 (126h14m58.470615688s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978204018 (126h14m58.47063647s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978677577 (126h14m58.471110026s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.978824227 (126h14m58.471256685s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.979375226 (126h14m58.471807678s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.980576318 (126h14m58.473008766s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.981861671 (126h14m58.474294113s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982175826 (126h14m58.474608269s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982521262 (126h14m58.474953707s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.982863112 (126h14m58.475295557s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983256721 (126h14m58.475689167s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.983616735 (126h14m58.476049178s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984050419 (126h14m58.476482867s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984431464 (126h14m58.476863913s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.984785095 (126h14m58.477217542s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985093505 (126h14m58.477525951s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985369715 (126h14m58.477802157s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:37.985687441 (126h14m58.478119897s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Uploading piece... Sep 3 20:22:38.515410987 (126h14m59.007843495s) [upload=111152]: Next returned nil segment Sep 3 20:22:38.515430046 (126h14m59.007862488s) [upload=111152]: Waiting for error group managing segments... Sep 3 20:22:38.719546875 (126h14m59.211979307s) [noise=true , address=104.36.82.62:4413 , piece_id=HGD64HA5BJ33CQJAY5QU3JK7HBXXT4Z3L7EWRTOPFK7H2ZVZWJHA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.738425850 (126h14m59.230858294s) [noise=true , address=45.91.116.5:4301 , piece_id=7XPAPGKGWE4PEZ6MGD3CN6TAWHX4IFENE4DQLCHVUK4EQDJNTESQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.753226326 (126h14m59.245658755s) [noise=true , address=104.36.82.52:4116 , piece_id=DJ6VLL5CH2ZPOGMTIJKVAYNEAQIM2UPEAZQ74DED3B3DPXR3CZ5A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.764287798 (126h14m59.256720232s) [noise=true , address=77.90.31.14:4673 , piece_id=KDU6BLBVXXM442KOD3TUJT457SVO325UEQLFONQVOAWVAUSN2LOA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.766210975 (126h14m59.258643407s) [noise=true , address=104.36.82.50:4032 , piece_id=J5KYMILSMV7VPEF23ZA46OABRDDTI5HXOYDNOWYMZRCKFNRZNKMA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.771228300 (126h14m59.263660734s) [noise=true , address=104.36.82.12:4171 , piece_id=EK2GOLZ7BMGLWUO5YVRMX7AKZ3SR5D3IBFBU2LPUJUMFBQWWFR3A, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.791093306 (126h14m59.283525744s) [noise=true , address=104.36.82.60:4353 , piece_id=ZSSL7D75TWOP4ZA3B3XZMZJNUX7QYTLTSTG2WPBE2Q5XWKTV3EEA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.834716534 (126h14m59.327148976s) [noise=true , address=77.90.31.12:4598 , piece_id=SDC64VSJD73Q7YLCBIYQS3IJUWYV3466B4O232ZT3KD3O6SNSA5Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.862074562 (126h14m59.354506994s) [noise=true , address=104.36.82.8:4981 , piece_id=DMUY2DLWERCAWPMAYFCVRE6R76NGA7ULIE3ZBB3EUGXEA2QGM22Q, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:22:38.863831850 (126h14m59.356264285s) [noise=true , address=77.90.31.19:4854 , piece_id=CBP7GQL4EWKCG7M725EYBJZLHX4WKADH4KKA7KCRMYQPUV6AQPEQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.961034856 (126h14m59.453467301s) [noise=true , address=104.36.82.23:4454 , piece_id=T2A3IM2QENMMIBGNIZ654J3G7XH7KIOY5JAWOHUMKEQMXRSZLVXQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.966039309 (126h14m59.458471738s) [noise=true , address=104.36.82.49:4998 , piece_id=3GQ2YMORZ537OB5TGOBBC7IDSXTVRC5SDIA5WXC6I2QVZ6KYFEKA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:22:38.997057592 (126h14m59.489490028s) [noise=true , address=77.90.31.18:4824 , piece_id=74JDWLA242VS5OPZPFG3GSMP2Y7GCSSTU6RDXJURIBZCEXKDVQNQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: <nil> Sep 3 20:23:37.450072191 (126h15m57.942504634s) [noise=true , address=104.36.82.5:4860 , piece_id=PDDIDNFIMEHUEHFSNXM4MTFAMDVNKSF6VDRSERNEBWU3R6NFVFSA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1Qr9fghoRVbAvNm9AMEHhhtc7JqHWBuZerBttg3KW3J3GcDjrE, address:104.36.82.5:4860): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450207350 (126h15m57.942639786s) [noise=true , address=104.36.82.5:4869 , piece_id=ODTMACKLJJ5ZG6LRG732QCP42OIXGDL7CR7K3K22FBVS2U2GQVKQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr, address:104.36.82.5:4869): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450278321 (126h15m57.942710754s) [noise=true , address=104.36.82.48:4591 , piece_id=UYVECENPZCY5IYXOJOMDOKTJFVWT7D5QU7NFGAYIQCBVOKWPGZWQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:12EH78DqzdSGuL9CD9yLuGczHLfeYHeZNaZDhvWwCnvFJYR3YE7, address:104.36.82.48:4591): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450366790 (126h15m57.942799218s) [noise=true , address=104.36.82.48:4571 , piece_id=CELVN72QGVKMDGSTEE743US3IMM5KALGZN4DJ7TNIHN6MIJFMXVQ, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:104.36.82.48:4571): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450513232 (126h15m57.942945675s) [noise=true , address=104.36.82.46:4499 , piece_id=ZQM7ITTALPWRRY6AHKPK26ZPHGJP4UHDAKC6M3EI5LZBXWULZHDA, seg_pos={0 0}, upload=111152]: Done uploading piece. err: ecclient: upload failed (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc, address:104.36.82.46:4499): protocol: context deadline exceeded; context deadline exceeded Sep 3 20:23:37.450559197 (126h15m57.94299164s) [seg_pos={0 0}, upload=111152]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 3 20:23:37.568980389 (126h15m58.061412821s) [upload=111152]: Done uploading segments... Sep 3 20:23:37.569029892 (126h15m58.06146232s) [upload=111152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 2.624732383s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [........X....X.XXX..X.XX..X.XXX.........XXX.XXX.XX.X........] duration: 2.645811838s FAIL (2025-09-03 20:24:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.64583863s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.......XXX.X.XXXXX.....X..X......XX......X......X....X......] duration: 2.392150859s FAIL (2025-09-03 20:24:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 3 20:19:59.160975149 (126h12m19.653407592s) [upload=111115]: Starting upload Sep 3 20:19:59.161046442 (126h12m19.653478887s) [upload=111115]: Done starting upload Sep 3 20:19:59.161324224 (126h12m19.653756652s) [upload=111115]: Uploading segments... Sep 3 20:19:59.161368922 (126h12m19.653801367s) [upload=111115]: Got next segment. Inline: false Sep 3 20:19:59.161402930 (126h12m19.653835373s) [upload=111115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 3 20:19:59.161407898 (126h12m19.653840341s) [upload=111115]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 3 20:19:59.441173706 (126h12m19.933606138s) [seg_pos={0 0}, upload=111115]: Begin upload segment... Sep 3 20:19:59.441232055 (126h12m19.933664487s) [seg_pos={0 0}, upload=111115]: Done begin upload segment. Sep 3 20:19:59.441262305 (126h12m19.933694751s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441307709 (126h12m19.933740164s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441642451 (126h12m19.934074896s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.441895673 (126h12m19.934328125s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442080526 (126h12m19.934512972s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442437922 (126h12m19.934870369s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442450277 (126h12m19.934882729s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442684853 (126h12m19.935117298s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442916393 (126h12m19.935348839s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.442933217 (126h12m19.935365669s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443153630 (126h12m19.935586074s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443448024 (126h12m19.935880474s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.443463986 (126h12m19.935896435s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450157309 (126h12m19.942589772s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.450753288 (126h12m19.943185734s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:19:59.451199029 (126h12m19.943631482s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Uploading piece... Sep 3 20:20:00.079826177 (126h12m20.572258645s) [upload=111115]: Next returned nil segment Sep 3 20:20:00.080369319 (126h12m20.572801776s) [upload=111115]: Waiting for error group managing segments... Sep 3 20:20:00.193969186 (126h12m20.686401628s) [noise=true , address=45.91.116.12:4227 , piece_id=RZ6ATSOLTGUH3R425NVAL4HTDG5Z4YEGZJBZW7EAXDICQXSEDYPQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.201432903 (126h12m20.693865341s) [noise=true , address=77.90.31.12:4589 , piece_id=HK2GI47RAAQWMSOKTTUJINMBEM46VWBGY7OLFHJPDTEXT7JAE6RA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.206800194 (126h12m20.69923263s) [noise=true , address=45.91.116.10:4082 , piece_id=TFSFTK5ECS3M4GEOSKMWUL26Y4ALPD6R4UDN5QBTKGY5F7TDCC2A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.210342618 (126h12m20.702775047s) [noise=true , address=45.91.116.4:4939 , piece_id=PL77HHI3ST53XBYJSDFP6Y65ODEF6YNNNTAQS535W6APPGQP7G5A, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.213337891 (126h12m20.705770327s) [noise=true , address=45.91.116.5:4298 , piece_id=YFYGEMAA5CPAWEBDQTOC4PTUHTLQDJ5XFG5ON2NVMLY24JEG2FSQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.228743352 (126h12m20.72117579s) [noise=true , address=77.90.31.16:4735 , piece_id=OJRSYQMB3MBKBRZHC3NQBRRB42IO33JPOLWMRX52RH6JD6T4T6JA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.229004233 (126h12m20.72143667s) [noise=true , address=45.91.116.7:4009 , piece_id=DWSPNYG3HHK5UR7PJ4XJID3QUTSJ7N4ATBKHYE2PPHRMH47H7GEQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.235842697 (126h12m20.728275139s) [noise=true , address=45.91.116.8:4048 , piece_id=2QIRGQFNSHVLMQLVPBGZI4M6TQH6HWKWM6R43GCBDWPYQDHFXCSA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.245052412 (126h12m20.737484865s) [noise=true , address=77.90.31.12:4592 , piece_id=I3GX53G37C573VDPYBLPO4LRAIKJ4LIPUUTZ3IWFHJFQ5Q3HZGXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.247624340 (126h12m20.740056796s) [noise=true , address=45.91.116.6:4973 , piece_id=4FSCB7CVE6WBGHETQ3M2423JSFLQREIOVN6ZINIJ6CVTDB5ZIA4Q, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.254021587 (126h12m20.746454039s) [noise=true , address=45.91.116.14:4191 , piece_id=35AZHG5L6IRUCFQH3DTVGWCHQBHCW6FSLCNDYFW2FFXFOYSQJ4OA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:00.290334141 (126h12m20.782766584s) [noise=true , address=45.91.116.5:4297 , piece_id=A6UZQRCH7N2ZHNWPBNDA6V5L2MPPFISH4AS2MYXP4IVLJA4EV4BQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: <nil> Sep 3 20:20:28.859703096 (126h12m49.352135532s) [noise=true , address=104.36.82.5:4837 , piece_id=BNIFRIJNVXB6QB3BBIYPW3TIET3AH2MY4GK2AT27CAMDRYUKNAXA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:104.36.82.5:4837): protocol: write tcp 139.59.144.22:58482->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859802423 (126h12m49.352234854s) [noise=true , address=104.36.82.9:4007 , piece_id=E4RJBDZ7KEP4BJA6N5QGSXYQGOMTKO7YFQNYZPWS2KYFK5HAV5HA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:42990->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.859878688 (126h12m49.352311123s) [noise=true , address=104.36.82.10:4079 , piece_id=LIM2MKZS4OTZX2XMRRLR635LYQCJLUZUSBCB642ENHNICQ24BLTQ, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:47454->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860049221 (126h12m49.352481653s) [noise=true , address=104.36.82.6:4876 , piece_id=3Z7YIPRC76RCXBWSRTUWWY5P75YP5OFIM2U5463ZHBZBLPZJ7TAA, seg_pos={0 0}, upload=111115]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 3 20:20:28.860083452 (126h12m49.3525159s) [seg_pos={0 0}, upload=111115]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 3 20:20:28.970913369 (126h12m49.463345815s) [upload=111115]: Done uploading segments... Sep 3 20:20:28.970993100 (126h12m49.463425529s) [upload=111115]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.392184708s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)